-- Logs begin at Wed 2022-08-24 08:01:52 UTC, end at Wed 2022-08-24 09:14:20 UTC. -- Aug 24 08:33:26.020524 dv-cinder-852211-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 08:33:26.031564 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 08:33:26.036563 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 08:33:26 2022] *** Aug 24 08:33:26.036814 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 08:33:26.036967 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 08:33:26.037109 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: nodename: dv-cinder-852211-5-iscsi Aug 24 08:33:26.037257 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: machine: x86_64 Aug 24 08:33:26.037400 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: clock source: unix Aug 24 08:33:26.037535 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: pcre jit disabled Aug 24 08:33:26.037660 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: detected number of CPU cores: 4 Aug 24 08:33:26.037794 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: current working directory: / Aug 24 08:33:26.037928 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: detected binary path: /usr/bin/uwsgi-core Aug 24 08:33:26.038070 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: your processes number limit is 47680 Aug 24 08:33:26.038233 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: your memory page size is 4096 bytes Aug 24 08:33:26.038373 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: detected max file descriptor number: 2048 Aug 24 08:33:26.038518 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: lock engine: pthread robust mutexes Aug 24 08:33:26.038837 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: thunder lock: enabled Aug 24 08:33:26.039171 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 08:33:26.039385 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 08:33:26.078187 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: Python main interpreter initialized at 0x55ff5f1629c0 Aug 24 08:33:26.078187 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: python threads support enabled Aug 24 08:33:26.078187 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: your server socket listen backlog is limited to 100 connections Aug 24 08:33:26.078187 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: your mercy for graceful operations on workers is 90 seconds Aug 24 08:33:26.079278 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 08:33:26.079278 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: *** Operational MODE: preforking *** Aug 24 08:33:26.079478 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: *** uWSGI is running in multiple interpreter mode *** Aug 24 08:33:26.079478 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: spawned uWSGI master process (pid: 93414) Aug 24 08:33:26.079732 dv-cinder-852211-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 08:33:26.080953 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: spawned uWSGI worker 1 (pid: 93427, cores: 1) Aug 24 08:33:26.080953 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: spawned uWSGI worker 2 (pid: 93428, cores: 1) Aug 24 08:33:26.081172 dv-cinder-852211-5-iscsi devstack@keystone.service[93414]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 08:33:30.636725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:33:30.636725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: from cryptography.utils import int_from_bytes Aug 24 08:33:30.637984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:33:30.637984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: from cryptography.utils import int_from_bytes Aug 24 08:33:30.642178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:33:30.642178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: from cryptography.utils import int_from_bytes Aug 24 08:33:30.643083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:33:30.643083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: from cryptography.utils import int_from_bytes Aug 24 08:33:30.693603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:33:30.693892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:33:30.694179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] command line args: [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:33:30.694547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:33:30.694853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ================================================================================ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:33:30.695006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:33:30.695155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:33:30.695296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] admin_token = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.695440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] command line args: [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:33:30.695592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:33:30.695731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_dir = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.695864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ================================================================================ {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:33:30.695992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] admin_token = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] config_dir = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_source = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] config_source = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] debug = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] debug = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.696922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.697936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_config_append = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_dir = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.698834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_file = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_config_append = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.699892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_dir = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700797 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.700928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.701898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.702951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.703159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] publish_errors = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.703159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.703541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.703742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.703742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.704088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.704288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.704485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.704713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.704713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] standard_threads = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.705192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.705389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.705389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.705845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] use_journal = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] use_json = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] use_stderr = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.706856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] use_syslog = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.707221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.707397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.707397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.707770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.707957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.707957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.707957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.707957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.708629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.708830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.709015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] publish_errors = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.709197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.709197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.709551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.709733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.709733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.710095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.710285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.710468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.710830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.710830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.711201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.711383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.711383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.711733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.711919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.712093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] standard_threads = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.712268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.712268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.712578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.712756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.712756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.713067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.713243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.713408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.713586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.713767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_journal = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.713945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.713945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.714268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_json = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.714433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.714608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_stderr = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.719542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.719762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_syslog = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.719921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.719921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:33:30.720319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.720982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.721896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.722166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.722547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.722725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.722866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.722999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.723894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.724915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.725044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.725177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.725316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.725453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.726091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.726259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.726463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.726694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.726871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.727924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.728954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.729931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.730106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.730259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.730458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.730617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.730960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731661 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.731867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.732918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.733814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.734105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.734243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.735961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.736906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.737728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.738994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.739491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.739697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.739697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.739956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.740914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.741898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.external = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.742985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.password = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] auth.token = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.743774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.744930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.745995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.746932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.747990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.748903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.749776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.750996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.751129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.751264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.753870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.753870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.754237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.754382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.754520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.754770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.754770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.755985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.756937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.757985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.758760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.759940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.760948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.761898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.external = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.762978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.763929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.password = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.token = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.764970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.765997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.766991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.767922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.768941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.769887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.770862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.771927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.772964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.773869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.774866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.775954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] role.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.776896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] role.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.777957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.778825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.779875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.780752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.781014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.781636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.781772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.781908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.781908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.782959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.783916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.784936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.785839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.786911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.787956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.788790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.connection = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.789905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.790960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.791770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.792915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.793179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.794073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.795147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.796156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.797855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.798552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.798552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.798891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.799223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.799544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.799852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.800257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.800679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.800998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.801319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.801625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.801934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.802557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.802878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.803200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.803514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.803829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.804141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.804456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.804843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.805194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.805521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.805823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.806141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.806453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.806816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.807138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.807643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.807943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.808256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.808569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.808881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.809278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.809537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.809870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.810188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.810495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.810817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.811185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.811526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.811690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.811898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.812047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.812187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.812330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.812662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.812978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.813284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.813586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.813889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.814194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.814510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.814837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.815198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.815515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.815821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.816136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.816449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.816761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.817066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.817382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.817682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.817994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.818312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.818640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.818986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.819360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.819729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.820079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.820395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.820700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.820862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.821044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.821366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.821671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.821973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.822279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.822600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.822944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.823292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.823560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.823870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.824172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.824483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.824775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.825079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.825401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:33:30.825639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:33:30.854466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.854760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.856918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.857211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.857502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.867261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.887057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.887303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.925688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.929802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.978333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.978629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.979832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:30.980127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.048659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.048892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.049023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.049187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.049422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.049616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.081446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.081749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.082011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.082347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.082689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.082958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.083244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.103488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.137940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.138203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.138357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.138587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.138836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.139054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.154939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.155277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.155593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.155916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.156835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.157161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.157467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.166691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.181258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.181511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.181690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.181907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.182127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.182338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.182552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.201096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.201359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:33:31.201656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.201857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:33:31.202145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.202409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.203543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.203812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.212998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.213367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.213706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.213856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.214094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.214235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.214366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.214366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.214619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.214619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.214954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:33:31.221985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.225178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.225467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.225756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.226040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.226315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.226569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.226753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.226901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.227045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:33:31.227186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.227491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.227657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.227790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:33:31.227908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.228023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.228218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.228626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.228960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.229286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.239646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.240011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.240321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.245152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.248019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.251007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.251292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.251596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.251988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.252357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.252513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.252786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.252940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.253229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.253599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.253963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.254321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.254712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.255098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.257667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.257964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.258250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.264791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.265086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.269988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.270300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.270593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.270912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.271199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.274903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.274903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.274903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.280419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.282818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.283193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.287938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.288323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.288685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.290181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.290503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.294453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.296108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.296493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.297798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.298104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.300922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.301303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.301670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.302035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.302349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.302597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.302998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.303356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.303711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.305548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.308883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.309199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.309464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.309757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.310043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.310335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.321022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.321442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.325677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.326422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.326772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.327047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.327315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.331295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.331730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.334953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.336377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.336572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.336790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.339508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.339800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.340061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.340218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.340360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.340650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.344384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.344790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.345157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.345647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.346015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.346388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.353000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.353394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.353679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.353966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.354250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.354564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.354891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.363892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.364313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.364663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.365012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.370598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.372388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.373854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.375805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.376172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.376881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.378341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.379332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.379653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.379872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.380085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.380260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.380453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.380604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.380922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.387087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.388587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.392137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.392437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.392727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.393018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.394184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.394580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.394980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.395350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.395719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.396255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.401882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.402079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.402079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.402079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.409473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.409766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.410054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.410333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.410622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.410931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.411210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.411480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.411762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.412043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.412328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.412590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.412865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.413126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.415038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.418081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.421468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.423079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.424664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.425040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.425411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.432406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.434054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.437838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.438231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.438600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.439016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.447559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.447980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.448200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ff5f1629c0 pid: 93428 (default app) Aug 24 08:33:31.448446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.451772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5b06b-49ad-425a-844b-1fc888fc4571 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.452061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5b06b-49ad-425a-844b-1fc888fc4571 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.452332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5b06b-49ad-425a-844b-1fc888fc4571 None None] PATH_INFO: `/v3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.456915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:33:31.457102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.457238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Wed Aug 24 08:33:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:31.458241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.458600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.459217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.459488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.460267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.460469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.460812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.461182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.461643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.462043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.462484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.462952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:33:31.497974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ff5f1629c0 pid: 93427 (default app) Aug 24 08:33:39.226078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.226741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.226741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.227983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 2/2] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:39.237355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.237913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.237913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.310110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: INFO dbcounter [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] Registered counter for database keystone Aug 24 08:33:39.332668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG oslo_db.sqlalchemy.engines [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93427) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:33:39.339478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writer thread running {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:33:39.526567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG passlib.handlers.bcrypt [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93427) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:33:39.527044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG passlib.handlers.bcrypt [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93427) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:33:39.585154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:39.586844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:39.727864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1/3] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 08:33:40.053705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.123112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: INFO dbcounter [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] Registered counter for database keystone Aug 24 08:33:40.144124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG oslo_db.sqlalchemy.engines [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:33:40.150968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writer thread running {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:33:40.273440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a92b0-96a5-4101-bd86-b6eff93df638 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.280808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.283315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.283315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.283315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.296313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.348201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:33:40.348447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb8291-fe31-44f8-9b64-73f9b58ccdff admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.358849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 3/4] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:33:40.365400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.388964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3f9f1-bafa-4ccc-81d6-e36d15d9451f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.396832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.400616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.400616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.400616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] PATH_INFO: `/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.402395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] RBAC: Authorizing `identity:get_project(project_id=a6994c6fdbe24b2e98be4db89dbc57c9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.474889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING keystone.common.rbac_enforcer.enforcer [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:33:40.475231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b353e7-c029-4b6b-b8e1-638942469dae admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.480133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 2/5] 10.222.0.64 () {56 vars in 1207 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9 => generated 344 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:42.449558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0542-bdfe-4f24-b0c8-532d08e92d05 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.450139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0542-bdfe-4f24-b0c8-532d08e92d05 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.450139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0542-bdfe-4f24-b0c8-532d08e92d05 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.454467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 4/6] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:42.464324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.464861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.465309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.499652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:42.502372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:42.516275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 3/7] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 08:33:42.788474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0542-bdfe-4f24-b0c8-532d08e92d05 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:42.801399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0542-bdfe-4f24-b0c8-532d08e92d05 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:42.805974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:42.806595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.806910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.807186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.815629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:42.817446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad382a-a45c-46be-afbe-861c80c47385 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:42.830252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 5/8] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Aug 24 08:33:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 08:33:42.835989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:42.845768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33358b1f-a223-4117-aba6-eb5ad1ff90f6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:42.850359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:42.851023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.851303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.851574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.852901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:42.854812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef93fa3-da32-4f46-bc91-7be1cce28d47 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:42.856418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 4/9] 10.222.0.64 () {56 vars in 1198 bytes} [Wed Aug 24 08:33:42 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:44.888757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eba1e-2eea-4ac4-95bd-0e60ff0d7a35 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.889490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eba1e-2eea-4ac4-95bd-0e60ff0d7a35 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.889490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eba1e-2eea-4ac4-95bd-0e60ff0d7a35 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.890621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 6/10] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:44.897960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.898533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.898926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.923451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:44.924764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:44.943227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3f73-8c9e-4f94-b4c1-e60e6b0e2f5a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.943516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3f73-8c9e-4f94-b4c1-e60e6b0e2f5a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.943785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3f73-8c9e-4f94-b4c1-e60e6b0e2f5a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.945146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 7/11] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:44.951013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.951283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.951616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.996251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG passlib.handlers.bcrypt [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93428) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:33:44.996668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG passlib.handlers.bcrypt [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93428) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:33:45.017599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 5/12] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:45.042773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:45.044389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:45.154952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 8/13] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:45.284228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.313840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.427549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.439867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe98e5-0c36-4967-b48f-87b7092eb131 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.480883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.481652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.482044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.482527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.497676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:45.509076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:45.510167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:33:45.511099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 6/14] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:45.517512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.568601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817070da-473e-4804-a1af-5d271b87ea10 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.571779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.616388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.617313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.617611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.617936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.650137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77261f-ffa0-42fd-92d2-cb148917c0ad None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.694596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.695346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.695752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.696200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.696975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:45.697445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.699251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.712694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.714640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.714826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 7/15] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:45.722880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.826249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 9/16] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:33:45.832751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.835940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.870732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba285f2d-c303-4e49-8bfa-a69e562c4961 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.910683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.911722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.912073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.912419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.913921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.915959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.918021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 8/17] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.925149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.952949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.959416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ef154-74c4-475a-bf65-3796fb96641d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.996183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.997071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.997354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.997699 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.020290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.021855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:33:46.045710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:46.047488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:33:46.049090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:33:46.049090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:46.049090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:33:46.049090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 10/18] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:46.055890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.091509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c9816-5717-4872-bd6c-db8a5d3af3ce None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.116141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.145980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.147160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.147663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.148139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.150356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.153194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.156716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 9/19] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:46.163193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.170939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcf1d2-d219-4ba8-8a06-b5718e661422 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.222230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.223243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.223683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.224138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.225225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:46.225727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.228085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537fbb4-4c5d-4d1c-8165-b19762e218f0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.252778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 11/20] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:46.278317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.303681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605377b3-2574-41db-9132-2603513fd4f4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.349291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.350020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.350367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.350761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.351633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.domain.id=default, user.id=9e24fe7edd574919ac552aa8b7b7cbf0) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:46.352006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.353600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416c4f-990e-4cfe-b23a-1f24cc547b31 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.362516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 10/21] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/role_assignments?user.id=9e24fe7edd574919ac552aa8b7b7cbf0&scope.domain.id=default&role.id=3f587a5233884fd6861867344e2c084b => generated 233 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:49.867828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68de3-8871-4d33-ac5d-a9d58a498964 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.869226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68de3-8871-4d33-ac5d-a9d58a498964 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.869917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68de3-8871-4d33-ac5d-a9d58a498964 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.872843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 12/22] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:49.882385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11b33-705a-4727-ac1f-21796156868d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.883296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11b33-705a-4727-ac1f-21796156868d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.883445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11b33-705a-4727-ac1f-21796156868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.920719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-83f11b33-705a-4727-ac1f-21796156868d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:49.922817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-83f11b33-705a-4727-ac1f-21796156868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.070861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 11/23] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:50.284450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573b29-1bac-4a62-81d8-7126b616996a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.285251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573b29-1bac-4a62-81d8-7126b616996a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.285932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573b29-1bac-4a62-81d8-7126b616996a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.288757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 13/24] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:50.298598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e6b-243d-4174-bdf3-076820cb86c5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.299137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e6b-243d-4174-bdf3-076820cb86c5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.299650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e6b-243d-4174-bdf3-076820cb86c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.348008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4b473e6b-243d-4174-bdf3-076820cb86c5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:50.350120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4b473e6b-243d-4174-bdf3-076820cb86c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.490803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 12/25] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:50.805866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503676-4e83-4a4c-8288-bd7e583235f2 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.806371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503676-4e83-4a4c-8288-bd7e583235f2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.806891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503676-4e83-4a4c-8288-bd7e583235f2 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.809373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 14/26] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:50.826114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797459c5-eab1-4c7f-9c2b-ee793b337890 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.827151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797459c5-eab1-4c7f-9c2b-ee793b337890 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.827885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797459c5-eab1-4c7f-9c2b-ee793b337890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.847314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:50.875441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-797459c5-eab1-4c7f-9c2b-ee793b337890 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:50.877470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-797459c5-eab1-4c7f-9c2b-ee793b337890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.885165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0503676-4e83-4a4c-8288-bd7e583235f2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.102289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0503676-4e83-4a4c-8288-bd7e583235f2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.121242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 13/27] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:51.141044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.156777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dc8e-d068-4100-97c3-99b8a4331b83 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.157436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dc8e-d068-4100-97c3-99b8a4331b83 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.163613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dc8e-d068-4100-97c3-99b8a4331b83 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.163613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 14/28] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:51.172749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.173457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.174076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.181157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.181829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.182115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.182387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.211357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:51.218253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:51.219951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:33:51.221715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 15/29] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:51.233462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.240859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:51.243067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:51.478001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 15/30] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:51.479960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.501824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade20796-516b-436e-9f71-50a0881f21ec None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.566701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.566701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.566701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.566701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.567384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:51.567534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:51.568970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:51.594619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 16/31] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:51.595604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.659205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.659461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.759972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6762e7c-9b3b-46be-9f49-0e490c6d3468 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.828018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633243e3-d205-445d-873c-5e720720a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.830401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633243e3-d205-445d-873c-5e720720a277 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.831216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633243e3-d205-445d-873c-5e720720a277 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.831997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633243e3-d205-445d-873c-5e720720a277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.858758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633243e3-d205-445d-873c-5e720720a277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:51.861862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633243e3-d205-445d-873c-5e720720a277 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:51.879103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba817b8a-ef25-44f7-883d-ca1444750c1d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-633243e3-d205-445d-873c-5e720720a277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:33:51.896644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:51.898147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:33:51.899765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:33:51.899765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:51.899765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:33:51.901306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 16/32] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:51.907705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633243e3-d205-445d-873c-5e720720a277 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.961828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.963513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.963809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.964097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.964818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:51.966342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.082626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633243e3-d205-445d-873c-5e720720a277 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.116318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.147981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.149088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.149872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.150669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.153006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.155922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170fc97-9c11-4b70-9198-ef840ea9c996 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.165476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 17/33] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.165476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae6b4-de36-46a3-9a7f-c211cd976124 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.165476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae6b4-de36-46a3-9a7f-c211cd976124 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.165476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae6b4-de36-46a3-9a7f-c211cd976124 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.165476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 18/34] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:52.166134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ae6b4-de36-46a3-9a7f-c211cd976124 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.210444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 17/35] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:51 2022] POST /identity/v3/roles => generated 225 bytes in 615 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:52.211275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.358940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.363718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af25c-92ff-462c-96cf-a0c73ffd6e27 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.369121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ae6b4-de36-46a3-9a7f-c211cd976124 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.415215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.416271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.416271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.416550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.417098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.418762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988c499-493e-4418-83a7-541f4b9a1820 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.426223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.427016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.427452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.427901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.429002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:52.429505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.432262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.459468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 19/36] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.460372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.503422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 18/37] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/roles => generated 219 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:52.505754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165746e1-6939-41f5-93ce-bdaf8d7c903e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.506231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165746e1-6939-41f5-93ce-bdaf8d7c903e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.506722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165746e1-6939-41f5-93ce-bdaf8d7c903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.524091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.541832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-165746e1-6939-41f5-93ce-bdaf8d7c903e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:52.545540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-165746e1-6939-41f5-93ce-bdaf8d7c903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:52.618800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76037821-73eb-424f-bc17-92cab8b27cf4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.651713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 19/38] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:52.659065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347609b6-3bfc-42b3-9f9c-384800c5c0f5 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.659065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347609b6-3bfc-42b3-9f9c-384800c5c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.659065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347609b6-3bfc-42b3-9f9c-384800c5c0f5 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.659065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 20/39] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:52.660100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347609b6-3bfc-42b3-9f9c-384800c5c0f5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.699616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.700867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.701332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.701812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.703969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.706912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc898-c600-447b-82d9-6554f52188fd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.710454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 20/40] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.714220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.714777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.715272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.752763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:52.754918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:52.864834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.885490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 21/41] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:52.885490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.898932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347609b6-3bfc-42b3-9f9c-384800c5c0f5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.967316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.968323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.969543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.970143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.970640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.972770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.975839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.078155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110bcd9-e535-4eea-9ba6-b331fca30a86 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.118964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 21/42] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/projects => generated 301 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:53.120780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:53.132030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.133571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.134332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.135074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] PATH_INFO: `/v3/domains/default/users/9e24fe7edd574919ac552aa8b7b7cbf0/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.144860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9e24fe7edd574919ac552aa8b7b7cbf0, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:53.144860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.280431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081b08-aa1b-4787-b351-cd76bddf4084 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.331940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.332854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.333304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.333726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.334596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:53.336818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.340782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 22/43] 10.222.0.64 () {58 vars in 1361 bytes} [Wed Aug 24 08:33:52 2022] PUT /identity/v3/domains/default/users/9e24fe7edd574919ac552aa8b7b7cbf0/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:53.340782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:53.407892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:53.416207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 22/44] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:53 2022] POST /identity/v3/roles => generated 223 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:53.582228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4f71-1af6-4084-8559-d5fb500bb6f5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.584466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:53.622508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.623726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.623726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.623726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.625057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:53.626887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe356c-3d3a-4244-a9fe-d9ca08507f3d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.636460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 23/45] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:53.638518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:53.763268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef708b54-ab99-4f30-9783-caf715105de0 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.799688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.800320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.800610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.800890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.801550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:53.803043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a01b2-97c7-491d-b6d0-29771acef7e1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.904020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 23/46] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:33:53 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:55.334481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad89ce-61b6-417b-9328-22950555f0e6 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.335038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad89ce-61b6-417b-9328-22950555f0e6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.335259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad89ce-61b6-417b-9328-22950555f0e6 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.336890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 24/47] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:55.343415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.344135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.344394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.365363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:55.366974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:55.418246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.439364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 24/48] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:55.711969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad89ce-61b6-417b-9328-22950555f0e6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.830012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad89ce-61b6-417b-9328-22950555f0e6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.867475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345daa93-9075-4442-be35-986053c96f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.868130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345daa93-9075-4442-be35-986053c96f90 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.868412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345daa93-9075-4442-be35-986053c96f90 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.868684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345daa93-9075-4442-be35-986053c96f90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.880349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345daa93-9075-4442-be35-986053c96f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-345daa93-9075-4442-be35-986053c96f90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:55.885364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:55.886320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:33:55.888111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 25/49] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:55.893149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.975617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6570b8-898a-42f7-bee3-9617428a8e72 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.975960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6570b8-898a-42f7-bee3-9617428a8e72 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.976410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6570b8-898a-42f7-bee3-9617428a8e72 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.978152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 26/50] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:55.987407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.987868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.988325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.009411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f872b-975a-4616-be23-ce99d7648521 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.016830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:56.019373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:56.054965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.056129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.057088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.057542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.057986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.059269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:56.059805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.061993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.073501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 25/51] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:56.082729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.115980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 27/52] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:56.136163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.206891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a287e67-2494-4c7a-87a8-9d3871011200 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.244568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.245134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.245422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.245700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.247670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.249158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.250837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 26/53] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:56.259689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.380681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d28bd8-dcb9-4eda-bf36-017b37bdcb66 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.403598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.426362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.427089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.427374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.427663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.428895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.430428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.432201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 28/54] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:56.438998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.478353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.561799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0130462-fe07-423f-89f8-b65f5f811ca2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.579844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf70b-0d27-44e7-a394-7486fc81d35e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.582988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.616579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.617418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.617814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.618261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.620053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.622535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.624409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 27/55] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:56.630689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.632566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.633288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.633895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.634268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.635217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.domain.id=default, user.id=9e24fe7edd574919ac552aa8b7b7cbf0) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:56.635630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.638716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cfdd8-0599-4910-b8b5-42b9f4233dc1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.644430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 29/56] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/role_assignments?user.id=9e24fe7edd574919ac552aa8b7b7cbf0&scope.domain.id=default&role.id=3f587a5233884fd6861867344e2c084b => generated 530 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:56.880731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520ec1-5607-4ba8-b672-266f295aa0c9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.922077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.922788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.923138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.923454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.924139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.925665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278854-5695-4105-b955-c89213aa3f02 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:57.017784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 28/57] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Aug 24 08:33:56 2022] POST /identity/v3/projects => generated 298 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:59.016882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495fc0e-b77f-4cbe-88a1-af9faeb9045c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.017447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495fc0e-b77f-4cbe-88a1-af9faeb9045c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.017447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495fc0e-b77f-4cbe-88a1-af9faeb9045c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.019299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 30/58] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:59.025311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.025849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.025984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.044992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:59.046214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:59.076400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.120537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 29/59] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:59.391754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495fc0e-b77f-4cbe-88a1-af9faeb9045c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.496163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.507207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495fc0e-b77f-4cbe-88a1-af9faeb9045c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.545933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.546594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.546922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.547201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.548395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.549873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19160d65-ae0f-424c-8446-5af57af1bafe None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.551564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 31/60] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:59.561084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.699399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03ba09-e880-4bb6-8def-1cb0b244d0c1 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.738011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.738636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.738957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.739237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.739898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.741422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aea2b-573c-4096-9896-d6c5cd880244 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.885014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 30/61] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:59 2022] POST /identity/v3/users => generated 275 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:03.420848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba90af-872e-4368-9547-3f15cd1e71a1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.421895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba90af-872e-4368-9547-3f15cd1e71a1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.422386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba90af-872e-4368-9547-3f15cd1e71a1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.424129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 32/62] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:03.430362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.431365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.431839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.452241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:03.453818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:03.470693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:03.536311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 31/63] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:03.852930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba90af-872e-4368-9547-3f15cd1e71a1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.990320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba90af-872e-4368-9547-3f15cd1e71a1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.035739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.036965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.037589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.038263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.052128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-040cbe8c-c5b0-4839-8947-01b0f0c2daf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:04.057427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:04.058517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.059822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:04.061762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 33/64] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:04.067902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.155692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8b0d-ec63-4cf2-a948-f5ff4ae46167 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.156337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8b0d-ec63-4cf2-a948-f5ff4ae46167 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.156847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8b0d-ec63-4cf2-a948-f5ff4ae46167 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.159237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 34/65] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:04.165252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d77b-a21f-410c-90dc-ce8df20d31d5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.165866 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d77b-a21f-410c-90dc-ce8df20d31d5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.166378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335d77b-a21f-410c-90dc-ce8df20d31d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.190950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6335d77b-a21f-410c-90dc-ce8df20d31d5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:04.192220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6335d77b-a21f-410c-90dc-ce8df20d31d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:04.209798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.215824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c803d-ad2b-4b6f-97ef-e2485ad11624 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.260807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.262081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.262374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.262688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.263708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:04.264075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:04.266340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f92d43-e229-4c2c-bae0-ff7e50ebe222 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:04.276268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 32/66] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:04.277838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278cd93-251f-41eb-8ca0-5617dec6f78c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.278114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278cd93-251f-41eb-8ca0-5617dec6f78c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.278574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278cd93-251f-41eb-8ca0-5617dec6f78c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.279848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 33/67] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:04.280852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc428d1-29d3-4558-a8b6-593bb3af7458 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.281279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc428d1-29d3-4558-a8b6-593bb3af7458 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.281730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc428d1-29d3-4558-a8b6-593bb3af7458 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.282994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 34/68] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:04.283747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc428d1-29d3-4558-a8b6-593bb3af7458 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.295800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 35/69] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:04.296936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc14e-288a-44a7-8862-2d5145a6b5aa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.297357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc14e-288a-44a7-8862-2d5145a6b5aa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.297644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc14e-288a-44a7-8862-2d5145a6b5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.299278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.318201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-229bc14e-288a-44a7-8862-2d5145a6b5aa None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:04.319762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-229bc14e-288a-44a7-8862-2d5145a6b5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:04.408497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc428d1-29d3-4558-a8b6-593bb3af7458 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.410877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 36/70] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:04.412532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.413025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.413461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.447377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.447915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:04.448921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:04.449506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.449995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.450468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.468387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:04.470616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:04.472625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 35/71] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:04.482923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.599822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 37/72] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:04.601045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.614360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.671148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.711513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542bafa-0a89-477d-a81f-d17627245a5f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.721959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f626c-56bd-4729-a5d7-0afcdebe2f3c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.750025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.751108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.751610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.752122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.759799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.760416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.760903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.761183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.764131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a106551e-953a-4da6-92b6-f25ea0f71af3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:04.769135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:04.770069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:04.770905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:04.774838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 38/73] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:04.777871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ff522-336e-4bf1-ae89-42e17740d5a8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.778266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ff522-336e-4bf1-ae89-42e17740d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.779220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ff522-336e-4bf1-ae89-42e17740d5a8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.780780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 39/74] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:04.781387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ff522-336e-4bf1-ae89-42e17740d5a8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.851591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:04.853401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:04.859671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.903237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ff522-336e-4bf1-ae89-42e17740d5a8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.927829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.944828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.945482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.945768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.946042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.958690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e6fe7349-e98b-4784-8882-2a544f28a2b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:04.963777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:04.964784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.966076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:04.966727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 36/75] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:04.966822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 40/76] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:04.966892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.974239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.974239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.974239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.992057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:04.993305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:05.071448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 41/77] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:05.072848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.087437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d6222-17b5-4524-9865-b087ee95cc13 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.144321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.145257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.145919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.146385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.147735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:05.148517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.150338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.162160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 37/78] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.163660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.164154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.205281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.213624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02f46-b176-4c55-8c26-80e69b258be3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.271621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.272674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.273191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.273705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.275170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:05.275727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.278816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.285979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e0c40-f343-4002-9a9d-92ca2aa999ec None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.296307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 42/79] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.297954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.327970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.328626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.328911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.329191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.330355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.331860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.338626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 38/80] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.338626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.439897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554587-93cd-45b9-913c-50b1644ff779 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.450628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.454639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a441-3313-4256-8d62-1da94561d93b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.467725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.479770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.480477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.480758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.481034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.491316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.491954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.492243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.496804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.496958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:05.497044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.497044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.497309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 39/81] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.498081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:05.498802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:05.500525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 43/82] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:05.502675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.615428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d75d3-2415-49b7-9059-55dd36641d48 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.618874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dcc2a-622e-4d4f-aacf-f113a639d477 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.651737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.652397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.652678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.652953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.654101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.655660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.656361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.657040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.657194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 40/83] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.657679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.658022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.658937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:05.659399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.659553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.661229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.668735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 44/84] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=4eed1433d0c54215a270109801b14d73 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.669373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.701659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.749463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.794539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fbb26-3837-42d9-a475-f90238262ed2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.825890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642f3f-262c-4d7a-836a-dc6f3d1874c7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.849567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.850698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.851258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.851707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.853617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.856762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.859874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 41/85] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.861129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.866468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.867460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.867832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.868123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.868839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:05.869148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.870556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.884610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 45/86] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.884610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.026905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd33a48-2745-4d8e-9cfa-78ac8a669d26 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.029389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.047372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32132f-62d7-48f9-ac5a-fb03bc3fb496 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.058430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.075335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.076312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.076737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.077204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.088392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.089036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.089340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.090205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.191998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.194461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.217152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.218688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.338859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 42/87] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:06.338859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.340794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.345328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 46/88] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:06.352824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.363273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.480970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec378a86-a8cf-4926-9f55-63ad66f45ba9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.485620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f2b2a-53bd-476f-9e44-f9ebf247eb0d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.522224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.522982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.523333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.523688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.524891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.525956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.541284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.542156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.542581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.543059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.544471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.546358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.548200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 47/89] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:06.549008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.613159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 43/90] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:06 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:06.619780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.647518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.680640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53489-0582-4ae8-adfb-b3f4df546026 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.736436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.756848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.759242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.760019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.761406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.765311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.768093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.778166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 48/91] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:06.778166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.804515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7285db-6830-49f6-9b05-039fcd6f27a5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.873304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.875061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.875575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.876123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] PATH_INFO: `/v3/projects/7df8efa7277d4453beaee4f2df2d1f09` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.877661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] RBAC: Authorizing `identity:get_project(project_id=7df8efa7277d4453beaee4f2df2d1f09)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.881426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.885792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 44/92] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/projects/7df8efa7277d4453beaee4f2df2d1f09 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:06.888550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.986405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570fb7-a449-4faf-a486-cfa880edf175 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.062060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.063188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.063684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.064179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.088339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa3cf-5a5b-4728-a796-2745412bf004 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.099251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.152954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.170863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.172698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.174550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.175660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.177332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=9e24fe7edd574919ac552aa8b7b7cbf0) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:07.178409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.182200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.202742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 45/93] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/role_assignments?user.id=9e24fe7edd574919ac552aa8b7b7cbf0&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:07.205253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.235785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.238693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.455254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 49/94] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:07.466511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.570144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.640022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde79ae2-6c04-41c6-ba85-31febe8fa800 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.697346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cb20-9a4b-4b66-9bfe-3d142df67229 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.774218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.775669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.776436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.777685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.779985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.783221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.787230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 50/95] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:07.790723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.790723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.790723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.791149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.792374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=7df8efa7277d4453beaee4f2df2d1f09, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:07.793512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.798897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.804575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fcf02-64c7-4e10-9500-aa54b57dbfd4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.819844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 46/96] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=7df8efa7277d4453beaee4f2df2d1f09&role.id=4eed1433d0c54215a270109801b14d73 => generated 259 bytes in 616 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:07.847183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a7bc-1313-4c88-bc34-273d2018e73f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.847183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a7bc-1313-4c88-bc34-273d2018e73f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.847710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a7bc-1313-4c88-bc34-273d2018e73f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.850566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 47/97] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:07.862696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.862696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.862696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.931589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:07.931589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:08.115300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.139005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214269-8c4f-42ac-a175-fa125269b915 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.172293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.184739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 48/98] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:08.242459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.247776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.248259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.248847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.250335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:08.250922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.253689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.268791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 51/99] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 259 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.970050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:09.161682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac4ef-3f78-4ed4-818e-c968a913de36 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:09.256082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:09.257667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.258469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.259251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.286726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:09.292990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:09.293970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:09.294805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:09.297300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 49/100] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:09.303407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:09.421295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f34c1a-85b3-4b60-9baf-1debea8d0241 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:09.461138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:09.462138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.462716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.463246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.464174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:09.464693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:09.466449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:09.477222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 52/101] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:09.483995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:09.560422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:09.639804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c45c8-6bde-4478-ab1c-fbcbb70f2b1b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:09.682591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:09.683733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.684234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.684721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.686172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:09.687912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:09.689896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 50/102] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:09 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:09.695703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:09.771014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:09.820539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d63e-0a8e-4c86-91f0-3a2a041838ad None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:09.857565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:09.858294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.858907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.859241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.860468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:09.861963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7bc6c-1bb9-4c30-84dd-169990167ba9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:09.863717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 53/103] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:09 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:09.869852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:09.986101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916aa2-2066-4fd6-b295-33f048f58e00 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.986528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916aa2-2066-4fd6-b295-33f048f58e00 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.986732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916aa2-2066-4fd6-b295-33f048f58e00 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.988545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 54/104] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:10.008168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ef71-6565-41d1-b0d5-6c09ed568f5e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.008493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ef71-6565-41d1-b0d5-6c09ed568f5e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.008728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ef71-6565-41d1-b0d5-6c09ed568f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.027472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f9754-a382-42e2-8755-a0f0c75f42cb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:10.040566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0038ef71-6565-41d1-b0d5-6c09ed568f5e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:10.041924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0038ef71-6565-41d1-b0d5-6c09ed568f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:10.058494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:10.105202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:10.105857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.106156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.106459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/5b57159c4512494f809abfa825aef1c8/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.108525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, user_id=5b57159c4512494f809abfa825aef1c8, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:10.110022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:10.142118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 55/105] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:10.201992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 51/106] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:09 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/5b57159c4512494f809abfa825aef1c8/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:34:10.631361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:10.632935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7681-c7e0-4fe3-b40f-1e63b6a63c09 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.634231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7681-c7e0-4fe3-b40f-1e63b6a63c09 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.635142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7681-c7e0-4fe3-b40f-1e63b6a63c09 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.637748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 56/107] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:10.652102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.652886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.653506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.686542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:10.689715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:10.729449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:10.779677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 57/108] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:10.848940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f563059-0139-45a9-8753-67a241e1eb92 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:10.895960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:10.896703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.897002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.897281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.898431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:10.899958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:10.901913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 52/109] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:10.907700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.049862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7191d0b-8a37-4718-afd5-38b419047039 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.091090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.091731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.092015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.092294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.092960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.094796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.107918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.169768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.195286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 58/110] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:34:10 2022] POST /identity/v3/projects => generated 302 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:11.249396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2d39-2eeb-4809-81a5-e7d404788f6f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.288345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.289004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.289295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.289578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.302540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-2ca3a943-e996-4d8f-a0ab-f9d4fb42c00f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:11.308748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:11.309709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:11.310492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 53/111] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:11.320148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.356915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.467037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171e8d9-529c-4ab1-8ed2-5fea150c9e96 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.494707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eca24-9bbb-426a-9d1c-addc050a4d36 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.495918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eca24-9bbb-426a-9d1c-addc050a4d36 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.496413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eca24-9bbb-426a-9d1c-addc050a4d36 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.498086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 54/112] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:11.514284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.515234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.515234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.515519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.516428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:11.516738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.517709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.518205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.518563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.519282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.528649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 59/113] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:11.534586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.553972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:11.556551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:11.687814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 55/114] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:11.711109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.742209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5c0c-ec00-4180-9b48-ab9a00a809a8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.784350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.785070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.785348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.785623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] PATH_INFO: `/v3/users/b290ba37e7e54f35b3ff5bc00c087224` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.806463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] RBAC: Authorizing `identity:get_user(user_id=b290ba37e7e54f35b3ff5bc00c087224)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.808472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08253bea-6b43-4439-9189-ead2ed5da486 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.810433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 60/115] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/users/b290ba37e7e54f35b3ff5bc00c087224 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:11.816234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.827507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031996d-81c3-451c-8aa0-5872691c58f4 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.827507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031996d-81c3-451c-8aa0-5872691c58f4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.827507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031996d-81c3-451c-8aa0-5872691c58f4 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.829382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 61/116] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:11.835364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.835868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.836421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.851973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.863248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:11.870895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:11.956617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf61314-c1d5-4ad8-b3b4-0721004b2261 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.019530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 62/117] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:12.031938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.032980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.033518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.034041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.124421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.126849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.162579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.258008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 56/118] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:12.264609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.341351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.343310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.433276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd424-a96f-43e5-b9ab-eb652e4c4161 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.486638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5fbd4-66f1-4b82-bc7b-ff3f859146c9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.536931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.549639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.550841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.551339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.551790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.564405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:12.568759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.569837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-19ed00cf-ef55-42cb-96d7-45e4ee72ea8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:12.570315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:12.571189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:12.571960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:12.572555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.572555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.578865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 57/119] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:12.580074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.580760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d14-2cef-4c6b-a072-c0549791d4bc None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.581191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d14-2cef-4c6b-a072-c0549791d4bc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.581621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d14-2cef-4c6b-a072-c0549791d4bc None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.582942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b95293-d9e2-4003-a53d-1edf7cafd594 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.583378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 58/120] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:12.585631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404d14-2cef-4c6b-a072-c0549791d4bc None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.591125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 63/121] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:12.593461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.593963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.594771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.635133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:12.637966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:12.756297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404d14-2cef-4c6b-a072-c0549791d4bc None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.761513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 64/122] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:12.767188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.823836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.824783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.826223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.826668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.827533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:12.829032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.830730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.859087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 59/123] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:12.860293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.917985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.978855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.987454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a560c43-8fd9-414a-9be1-d95eb70b0554 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.026238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.026842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.027129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.027400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.028165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=b290ba37e7e54f35b3ff5bc00c087224) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:13.028471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.029964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.036258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 65/124] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/role_assignments?user.id=b290ba37e7e54f35b3ff5bc00c087224&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.042705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.071898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d28473-4e7a-410f-9293-c0955a5ee640 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.163552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.166442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.167666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.168179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.193988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:13.208542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:13.209589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:13.210405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:13.212662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 60/125] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:13.214162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.274048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44aef-c095-40be-90e1-adc002f324df None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.403605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.408139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.409298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.409803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.410303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.412707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.415606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.428328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 66/126] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.438097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.543162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357ccc-2967-4575-902c-7e00dd422f74 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.605184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.680799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.681813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.682274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.682807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.727703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:13.737494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:13.738559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:13.739398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:13.740006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950ddc4-fa4b-4c6f-928f-7631e9af3afe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.751228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 61/127] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 537 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:13.752185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.822028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b16f1-2226-4eae-9818-49690106888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.824300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b16f1-2226-4eae-9818-49690106888e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.824881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b16f1-2226-4eae-9818-49690106888e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.825232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b16f1-2226-4eae-9818-49690106888e None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.826234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b16f1-2226-4eae-9818-49690106888e None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:13.826856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b16f1-2226-4eae-9818-49690106888e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.828349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b16f1-2226-4eae-9818-49690106888e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.840693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 67/128] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.843819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187906-c535-4033-b991-43e66d9c635f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.844940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187906-c535-4033-b991-43e66d9c635f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.845829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187906-c535-4033-b991-43e66d9c635f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.848139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 68/129] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:13.848967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187906-c535-4033-b991-43e66d9c635f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.919865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4f1c-883c-411f-a36a-d9958a82bca2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.930784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.981927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.983038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.983594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.984105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.986181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.986878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187906-c535-4033-b991-43e66d9c635f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.988412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.991284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 62/130] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.035861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.037367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.037886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.038704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.039634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:14.040169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.042519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c08f3-7468-411e-8353-3734c71d419c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.054338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 69/131] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.056177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.056688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.057166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.071059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.079088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:14.084642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:14.168742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 70/132] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:14.170071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.185569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800549-9462-430b-b421-4f1ff41aae5b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.234303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.286407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.287325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.287617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.287901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.289774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.291717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.294021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 63/133] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.294861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.295261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195c7c-89ad-4c3d-bd61-29d453384929 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.337805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.338876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.339342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.339841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/9e24fe7edd574919ac552aa8b7b7cbf0/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.342214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, user_id=9e24fe7edd574919ac552aa8b7b7cbf0, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.344096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0fae7-3c6c-4903-96dc-9488dc0806b5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.424200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e8a43-0ac1-45ec-9b72-950bc70e554d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.435455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.471072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 71/134] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:14 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/9e24fe7edd574919ac552aa8b7b7cbf0/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:14.474319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931f25-8607-4918-b3b6-be7c149b5ce0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.474949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931f25-8607-4918-b3b6-be7c149b5ce0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.475457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931f25-8607-4918-b3b6-be7c149b5ce0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.477247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 72/135] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:14.481833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.482381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.482960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.483458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.483870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af931f25-8607-4918-b3b6-be7c149b5ce0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.485809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.489450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b8405-cc8c-40a4-a64a-2245e8a7c667 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.494321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 64/136] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.502680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.502680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.502680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.523192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:14.524551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:14.661361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 65/137] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:14.663709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.708103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af931f25-8607-4918-b3b6-be7c149b5ce0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.727277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.752577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.753146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.753432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.753704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] PATH_INFO: `/v3/projects/7df8efa7277d4453beaee4f2df2d1f09` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.754967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] RBAC: Authorizing `identity:get_project(project_id=7df8efa7277d4453beaee4f2df2d1f09)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.756563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.758623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 73/138] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/projects/7df8efa7277d4453beaee4f2df2d1f09 => generated 312 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.759537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.804548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f018be7-eb84-447e-affc-52359ffd46d3 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.813492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.858581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.859304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.859597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.859875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.864664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.864664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.869576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 66/139] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.871764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.911533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abaab1-ccaa-4061-a333-5a381594627f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.023220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.024272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.024804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.025363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.065112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:15.074056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caed30b-982d-4d83-97b5-7a2906aea0aa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:15.079689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:15.080611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:15.081383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:15.082438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 74/140] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:15.083720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.128447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.129133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.129414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.130330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] PATH_INFO: `/v3/projects/7df8efa7277d4453beaee4f2df2d1f09/users/5b57159c4512494f809abfa825aef1c8/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.132766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] RBAC: Authorizing `identity:create_grant(project_id=7df8efa7277d4453beaee4f2df2d1f09, user_id=5b57159c4512494f809abfa825aef1c8, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.134238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.160425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.226490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 67/141] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:14 2022] PUT /identity/v3/projects/7df8efa7277d4453beaee4f2df2d1f09/users/5b57159c4512494f809abfa825aef1c8/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:15.228506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.238371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e544d83-4c33-4b9b-a9e0-2f8cea615a90 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.286362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.290900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.292514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.293019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.293444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/5b57159c4512494f809abfa825aef1c8/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.297068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, user_id=5b57159c4512494f809abfa825aef1c8, role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.298850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.386385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 75/142] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:15 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/5b57159c4512494f809abfa825aef1c8/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:15.392074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.413988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5010a9-bee5-47fe-8805-cc02ab84b32a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.501173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.502176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.502569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.503050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.504349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.505970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a36fa-11ad-43fb-a3ea-cbe78405a9a9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.508231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 68/143] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:15.509608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d3c9-d051-48b8-b3ab-9d704df77894 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.509901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d3c9-d051-48b8-b3ab-9d704df77894 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.510773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d3c9-d051-48b8-b3ab-9d704df77894 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.512180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 69/144] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:15.531298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d3c9-d051-48b8-b3ab-9d704df77894 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.604500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.608232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7812cc-b752-4b2d-a05b-df869788924f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.686202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.686953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.687233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.687509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.689770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:15.690075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.695450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deea7ad-d90f-413f-8f88-551018dc8f38 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.718829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 76/145] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:15.720646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.721956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.722785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.766001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:15.767683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:15.775953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d3c9-d051-48b8-b3ab-9d704df77894 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.777846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.846641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.847303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.847590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.847869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.850034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.864551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 77/146] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:15.865368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.964408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 70/147] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:15 2022] POST /identity/v3/users => generated 283 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:34:16.027141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:16.035120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cdd43-75c0-471d-a537-1dd70f915afb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:16.087182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:16.087845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:16.088128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:16.088457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:16.091740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:16.093329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:16.095420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 78/148] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:16.101452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:16.261732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3768ec-f662-4e08-b778-0e0db49c70f9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:16.300577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:16.301588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:16.302104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:16.302579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:16.341303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:16.392942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:16.394859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:16.516747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 71/149] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:16.524782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:16.544740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:16.705521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:16.801929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30481e-71a0-4b31-869b-97bbe3c5cfd5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:16.820419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ee58-e948-4121-80d6-2abe302f4688 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:16.927281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:16.929068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:16.929876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:16.930670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:16.933333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:16.936577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:16.937009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:16.938051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:16.938514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:16.939016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:16.941684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 79/150] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:16 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:16.949300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:16.979746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:16.984690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:16.986295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:16.987784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 72/151] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 442 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:16.991787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:17.035818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:17.136802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8eaa-aa0e-439b-8dec-8c8d86ecb08b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:17.193708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172edae-816b-4180-9d36-f78b0d8d235d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:17.202329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:17.203893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.204723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.205504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.207324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:17.208197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:17.214509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:17.215664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1f99a-1144-4ab8-bf17-686208a33cc7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:17.234037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 80/152] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:16 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=4eed1433d0c54215a270109801b14d73 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:17.237253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20512b-2816-47fc-bad7-fcfb867472a4 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.238116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20512b-2816-47fc-bad7-fcfb867472a4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.239382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20512b-2816-47fc-bad7-fcfb867472a4 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.242853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 81/153] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:17.245916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:17.246560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.246863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.247146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.247851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:17.248154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:17.249687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:17.262151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.263074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.263820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.264292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 73/154] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:17.272908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:17.300747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:17.304367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:17.477612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 82/155] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:17.571427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7719-565c-4b6e-8a02-7ee1f48d2b65 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:17.646314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:17.650804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.650804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.650804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] PATH_INFO: `/v3/users/b290ba37e7e54f35b3ff5bc00c087224` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.651198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b290ba37e7e54f35b3ff5bc00c087224)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:17.653879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:17.656856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 74/156] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:17 2022] GET /identity/v3/users/b290ba37e7e54f35b3ff5bc00c087224 => generated 289 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:17.666861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:17.719273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:17.832204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b3ef7-d35d-42b9-902b-006e28c4483f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:17.876042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:17.888864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:17.889520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:17.889796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:17.890058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:17.891480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:17.892954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:17.897156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 83/157] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:17 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:17.907844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:17.993567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:18.089968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295595c6-f91d-4c03-8bb7-4185694803a4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:18.139748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:18.140412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.140695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.140972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.157134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:18.166026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:18.168335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:18.169796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 75/158] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:18.182546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:18.195090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:18.202195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ff36c-cc6d-4094-b763-693eafa19010 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:18.310384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:18.311569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.312060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.312552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/b290ba37e7e54f35b3ff5bc00c087224/roles/e84585ece55148caad35c7fe5dca9d5f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.320083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, user_id=b290ba37e7e54f35b3ff5bc00c087224, role_id=e84585ece55148caad35c7fe5dca9d5f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:18.323440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:18.358406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134b485-9e89-45d7-bfc6-f4e26b041d15 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:18.371614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:18.406178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:18.406889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.407149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.407387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.408083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:18.408389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:18.409852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:18.419769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 84/159] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:17 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/users/b290ba37e7e54f35b3ff5bc00c087224/roles/e84585ece55148caad35c7fe5dca9d5f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:18.424047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 76/160] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:18.436644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:18.716347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d724d2-6ff3-44f4-a797-834440fc0960 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:18.787731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:18.788907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.789471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.789959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] PATH_INFO: `/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.792931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] RBAC: Authorizing `identity:get_user(user_id=9e24fe7edd574919ac552aa8b7b7cbf0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:18.795800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065a0e5-873c-4ce8-9145-474d25cf46ad None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:18.799292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 85/161] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:18 2022] GET /identity/v3/users/9e24fe7edd574919ac552aa8b7b7cbf0 => generated 247 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:18.805318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:18.944005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b14a1c-7458-40d9-94fd-212a3a098a11 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:18.986617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:18.987602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.988083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.988553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.040116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.078257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.079829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.212417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 77/162] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:19.219143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.220807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbe6b-b0aa-42be-bc24-f5ee91e14b66 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.222153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbe6b-b0aa-42be-bc24-f5ee91e14b66 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.222977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbe6b-b0aa-42be-bc24-f5ee91e14b66 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.230455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 86/163] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:19.240781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e821a3-86c7-41ac-b009-0c19ad195c65 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.241720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e821a3-86c7-41ac-b009-0c19ad195c65 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.242589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e821a3-86c7-41ac-b009-0c19ad195c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.289854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-01e821a3-86c7-41ac-b009-0c19ad195c65 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:19.292927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-01e821a3-86c7-41ac-b009-0c19ad195c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:19.323524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.335035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.367328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2824-9833-4a87-bf5b-e897e7146b19 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:19.406234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:19.407137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.407422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.407792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.409083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.410568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.412315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 78/164] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:19 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:19.424746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.500894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 87/165] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:19.569971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bac127-240f-487e-bd80-54f0757806f1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:19.608054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:19.608925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.609353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.609771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.610675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=9e24fe7edd574919ac552aa8b7b7cbf0) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:19.611152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.612782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738f41-544f-4b5b-bf82-93d59fdc88c6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.619515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 79/166] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:19 2022] GET /identity/v3/role_assignments?user.id=9e24fe7edd574919ac552aa8b7b7cbf0&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:19.907826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df032a-7e7e-4039-ba24-4cab37d1d9a8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.908335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df032a-7e7e-4039-ba24-4cab37d1d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.908726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df032a-7e7e-4039-ba24-4cab37d1d9a8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.910587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 88/167] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:19.917881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.918727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.919520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.931219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.958465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:19.960739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:20.102091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 80/168] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:20.102991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df032a-7e7e-4039-ba24-4cab37d1d9a8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.365620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df032a-7e7e-4039-ba24-4cab37d1d9a8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.446446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.449870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.451789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.452508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.485217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:20.496174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:20.498290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:20.500087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:20.503665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 89/169] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:20.514015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.570839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.595136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.807676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874f4e-eb78-402d-8aac-4d8a1353e4c7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.818042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550993c-64ec-4c64-8fa7-64c4ac6ec51a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.873593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.885367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.887093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.888104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.888104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.889354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:20.889916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.892818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbcbf5-97b9-45e2-95a7-e3793413d35c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.919152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 81/170] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:20.921777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbeaa-4e94-4aec-83b0-064d15642d7f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.922270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbeaa-4e94-4aec-83b0-064d15642d7f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.922793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbeaa-4e94-4aec-83b0-064d15642d7f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.925635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 82/171] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:20.932602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbeaa-4e94-4aec-83b0-064d15642d7f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.935440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.936103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.936390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.936639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.964172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1e0b0916-e7d5-4617-bb14-0fefb3f2d7ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:20.969369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:20.970973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:20.972307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 90/172] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:20.976309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.976625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.976858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.027244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:21.028471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:21.122246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbeaa-4e94-4aec-83b0-064d15642d7f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.177428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.178524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.179064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.179637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.181929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.185393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.188164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 83/173] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:21.189323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.189648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:21.196259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 91/174] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:21.200762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:21.359130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.474790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4746-4d93-4cdd-b85e-d3acf38ec79b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.490984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbea5b-c52d-4e1f-9f90-e2f3ae056642 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.578371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.581089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.582352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.583219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.608828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.609933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.610446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.610986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.612305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:21.612860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.623973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597524-64c5-47ef-a01b-0a4e1bc9e6d3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.644964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 84/175] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:21.645762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1a130-d321-45f2-9687-42c3db2b9090 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.647836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1a130-d321-45f2-9687-42c3db2b9090 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.647836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1a130-d321-45f2-9687-42c3db2b9090 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.647836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 85/176] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:21.648563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1a130-d321-45f2-9687-42c3db2b9090 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:21.718507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.724941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.727525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.790138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1a130-d321-45f2-9687-42c3db2b9090 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.833832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.852760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.853517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.853858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.854204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.875666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3024b48a-6873-4f8b-8101-012366c21f5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:21.885444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:21.887222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 86/177] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:21.888782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5774ba-363b-4415-a924-96dafd9b273f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.890155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5774ba-363b-4415-a924-96dafd9b273f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.890155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5774ba-363b-4415-a924-96dafd9b273f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.890443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 87/178] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:21.898725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6353ee-7847-440c-a7d2-49c677f427bd None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.898725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6353ee-7847-440c-a7d2-49c677f427bd None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.898725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6353ee-7847-440c-a7d2-49c677f427bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.921056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-dd6353ee-7847-440c-a7d2-49c677f427bd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:21.922873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dd6353ee-7847-440c-a7d2-49c677f427bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:21.928624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 92/179] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:21.930013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.041764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 88/180] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:22.043954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.044493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.045124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.078081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:22.080159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:22.099073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.108843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0e51-147b-4819-9623-594caa7a645f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.184650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.186146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.186938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.187662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] PATH_INFO: `/v3/users/5b57159c4512494f809abfa825aef1c8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.189966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] RBAC: Authorizing `identity:get_user(user_id=5b57159c4512494f809abfa825aef1c8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.193055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.196438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 93/181] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:21 2022] GET /identity/v3/users/5b57159c4512494f809abfa825aef1c8 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:22.198255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.215912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 89/182] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:22.217556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.308086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.395706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330f31-9487-483f-8148-eb99a8ea6320 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.415615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d16993-affb-4fc0-b82f-186b4c3790d5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.463816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.465329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.466130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.466928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.468444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:22.469527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.472507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.482965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.484163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.484685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.485749 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] PATH_INFO: `/v3/projects/7df8efa7277d4453beaee4f2df2d1f09` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.487428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] RBAC: Authorizing `identity:get_project(project_id=7df8efa7277d4453beaee4f2df2d1f09)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.490202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.492793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 94/183] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:22.499494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 90/184] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/projects/7df8efa7277d4453beaee4f2df2d1f09 => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:22.499494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.500405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.560805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.676993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247cf82-56b9-46ca-ad76-8b9135485dc9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.684981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160b31-a5f2-4991-8032-0aa95a3a340c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.701745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.725552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.726530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.727008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.727456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.737649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.738599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.739084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.739534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.747179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:22.752443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:22.754269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 95/185] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:22.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.850833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.852860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.887897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.906192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611d677-a23e-4651-ae9e-9f556c387954 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.949370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.950023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.950291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.950551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.966312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.968075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.970374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 96/186] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:22.977819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.977982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 91/187] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:22.978183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.997120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.099666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195224-4686-48a3-917d-2e7663f9611d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.116751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61472e11-bd43-49d0-999d-0cd0ac177727 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.142289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.143060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.143483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.143844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.145182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.146882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bb225-17e4-455f-827d-0a8ca8b7cd0d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.148740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 92/188] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:23.150892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc361-d7c3-4d1b-96e5-fac518078ffa None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.151301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc361-d7c3-4d1b-96e5-fac518078ffa None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.151706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc361-d7c3-4d1b-96e5-fac518078ffa None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.153355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 93/189] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:23.154415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc361-d7c3-4d1b-96e5-fac518078ffa None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.169513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.170482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.170970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.171411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.173208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=7df8efa7277d4453beaee4f2df2d1f09, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:23.173724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.176719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.198786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 97/190] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=7df8efa7277d4453beaee4f2df2d1f09&role.id=4eed1433d0c54215a270109801b14d73 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:23.198786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.219539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.298933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.336258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc361-d7c3-4d1b-96e5-fac518078ffa None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.346321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3bcd5-db53-4595-a45e-dde4d9c59b53 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.399108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.399702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.400211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.400544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.401675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:23.402219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.403459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.415364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.416344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.416836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.417410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.420507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 94/191] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:23.422820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.544001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.546596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639adcac-98bf-49fe-bcdf-3614e8671e26 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.566024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.586203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376d18-2da2-4408-847a-07e9f9fb49a4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.637207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.638010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.638410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.638825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.645245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.645245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.645245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 95/192] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:23.645245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.668203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.698050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 98/193] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:23.699787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.700250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.700704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.721818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:23.723638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:23.784484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa54d-5e11-44fb-96be-4bcc261746d4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.801945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 99/194] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:23.803058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.834525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.835523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.835976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.836422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.837365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.839798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.869774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.934039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 96/195] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:23 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:23.935594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.959393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a05eda-6492-43cd-964c-86c100e6d4af None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.035367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.036371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.036794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.037227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.040034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.043567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.046822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 100/196] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:24.048979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.081099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.128030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ff56-3074-4abc-82b4-6f0b5cba2a4d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.177005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.178006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.178452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.178906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.180374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=5b57159c4512494f809abfa825aef1c8) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:24.180702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.183343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.193243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 97/197] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/role_assignments?user.id=5b57159c4512494f809abfa825aef1c8&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:24.194406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.256663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb91d-02a2-41c7-beab-ddf288a9e768 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.274076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.338594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.339828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.340309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.341179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.345649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.348384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.351962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 101/198] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:24.355938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.393150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fc573-5676-4973-b6ed-25173a39b2a5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.438274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.453789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.454136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.454425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.454843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.542678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac159a7-07c6-417f-a018-c81b2bda3d15 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.583867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.584530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.584786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.585066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.588305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.591105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.598687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:24.604019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:24.604942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:24.605746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 102/199] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:24.606391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.654354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.732821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd302c-3d1e-40cd-89cb-1925ade8aeb0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.773371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.774310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.774674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.775508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.776306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=a22e9598b27440c5aac9c2617ad8f86e) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:24.776929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.778562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.780410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 98/200] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:24.782104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.785570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 103/201] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/role_assignments?user.id=a22e9598b27440c5aac9c2617ad8f86e&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:24.791835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.842526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.991165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e1a4-c9aa-47fa-ad2a-87c1fb6cdb38 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.055109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.061456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9159-6218-4bde-a48d-cd6a001507ce None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.075255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.076288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.076682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.077185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.078453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:25.079041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.081472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.113330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 99/202] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.124643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.164552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.165675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.166185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.166690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.176419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.178124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.180808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 104/203] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.189083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.299281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.315657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823503-3d02-4161-8988-fab8866b4f1c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.339917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e70a-2852-4b7e-8ee9-d01a0a94fa7f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.371373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.372090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.372373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.372650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] PATH_INFO: `/v3/users/b290ba37e7e54f35b3ff5bc00c087224` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.375869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] RBAC: Authorizing `identity:get_user(user_id=b290ba37e7e54f35b3ff5bc00c087224)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.378366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.380219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 100/204] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/users/b290ba37e7e54f35b3ff5bc00c087224 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.387634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.430406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.431583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.431861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.432130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.432875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=a22e9598b27440c5aac9c2617ad8f86e) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:25.433179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.434646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.451696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 105/205] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/role_assignments?user.id=a22e9598b27440c5aac9c2617ad8f86e&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.538761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71880d9b-c0f7-41b0-bcce-56b637668401 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.580980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.581522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.581807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.582078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.623370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.673558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.675124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.844157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 101/206] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:25.850347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.913662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.020267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743ba30-5795-4588-a3a6-26da371d1650 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.065901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.066551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.066861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.067134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.068311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.069844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0f8e5-e967-487f-9205-1aeb67e79406 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.071909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 106/207] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:26.077846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.151901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.208825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3edfa6-74d5-40cd-9f54-91d6ee931f11 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.256142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.256769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.257052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.257329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.258077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, scope.project.id=5d21b37c9bd744349387927fa9dbae25, user.id=b290ba37e7e54f35b3ff5bc00c087224) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:26.258385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.259922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed55ce3-659b-42f1-872b-399e2971648f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.266614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 102/208] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/role_assignments?user.id=b290ba37e7e54f35b3ff5bc00c087224&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:26.762935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f04d-fffa-48a5-832c-7dcba2584ee8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.763261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f04d-fffa-48a5-832c-7dcba2584ee8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.763478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f04d-fffa-48a5-832c-7dcba2584ee8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.765370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 107/209] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:26.770783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d24b7-a046-443f-8c97-e2d2392e03b3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.771393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d24b7-a046-443f-8c97-e2d2392e03b3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.771874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d24b7-a046-443f-8c97-e2d2392e03b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.797254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-315d24b7-a046-443f-8c97-e2d2392e03b3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:26.799053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-315d24b7-a046-443f-8c97-e2d2392e03b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:26.891706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 103/210] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:27.070044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701762ff-d5d1-4831-93f7-34bc9cafa414 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.070044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701762ff-d5d1-4831-93f7-34bc9cafa414 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.070044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701762ff-d5d1-4831-93f7-34bc9cafa414 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.070044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 108/211] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:27.076479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.077062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.077545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.098228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:27.099852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:27.155558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.175010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 104/212] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:27.186053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701762ff-d5d1-4831-93f7-34bc9cafa414 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.294856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.299384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701762ff-d5d1-4831-93f7-34bc9cafa414 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.337908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.338480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.338793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.339073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.340214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.341706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf2679-2c3b-421e-b084-5bebe52bf15b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.343615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 109/213] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:27.349214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.428752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c8af-2724-4b81-84c8-3fd5d4a9bf0a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.429077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c8af-2724-4b81-84c8-3fd5d4a9bf0a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.429298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c8af-2724-4b81-84c8-3fd5d4a9bf0a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.431364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 110/214] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:27.437940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.438555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.439068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.461055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:27.462284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:27.480711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbbc3c-0b09-42ff-a566-447ae76c4064 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.525575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.526205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.526486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.526802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.527564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.528982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98494638-6fea-4828-9a85-99651a33a523 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.554350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 111/215] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:27.558948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.650751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 105/216] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:27 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:27.674088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.679227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf155c-04a8-45c7-ba10-240daddd75d4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.683000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a856-9b8c-4817-81d6-362897c083fe None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.688935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a856-9b8c-4817-81d6-362897c083fe None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.688935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a856-9b8c-4817-81d6-362897c083fe None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.688935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 106/217] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:27.691248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a5358-41ce-4748-9913-9744852463dc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.691533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a5358-41ce-4748-9913-9744852463dc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.691810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a5358-41ce-4748-9913-9744852463dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.715950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-891a5358-41ce-4748-9913-9744852463dc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:27.717849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-891a5358-41ce-4748-9913-9744852463dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:27.723088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.723814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.727487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.727487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.738283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:27.744909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:27.747133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:27.748266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 112/218] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:27.752388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.788207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.838154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 107/219] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:27.839313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a5358-41ce-4748-9913-9744852463dc None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.950727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e5c36-9721-4248-93e1-cd5935a770ef None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.026587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a5358-41ce-4748-9913-9744852463dc None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.030550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.031643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.032140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.032684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.033904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.034535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.038060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.059207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 113/220] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:28.065311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.092737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.093792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.094222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.094510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.097570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.114981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:28.125340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:28.126943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:28.128160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 108/221] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:28.130122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.183735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.220538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d227d-2a43-4387-95bb-0de6d0eee6e1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.264216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afee985-410d-4640-b98b-d040d5916a6e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.288970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.290325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.290835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.291110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] PATH_INFO: `/v3/users/6a12010a529b48a4b7c8a47a3648f2a1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.309507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] RBAC: Authorizing `identity:get_user(user_id=6a12010a529b48a4b7c8a47a3648f2a1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.311468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.313930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 114/222] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users/6a12010a529b48a4b7c8a47a3648f2a1 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.315164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.316076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.317029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.317990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.318705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.339102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:28.345975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:28.347185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:28.348045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 109/223] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:28.348908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.444918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.470462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.482477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc82a-f5d2-4d42-bac4-3381c2b4f46f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.485715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c3332-e7bf-47bf-8833-9f26cf8d16de None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.523379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.524019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.524374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.524734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.557882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.559600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.560600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.561190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.562697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.563581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.567201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.585861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 115/224] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.588308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.611404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.613087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.721952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.733303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 110/225] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:28.734076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.793714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c49f-5eef-434d-b004-583ceae88edf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.850187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc264fa-b0f4-4f97-9d9f-3b15ec2d8b34 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.854918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258413-372d-42df-aae1-70f77e690650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.855550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258413-372d-42df-aae1-70f77e690650 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.855859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258413-372d-42df-aae1-70f77e690650 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.856107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258413-372d-42df-aae1-70f77e690650 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.856775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258413-372d-42df-aae1-70f77e690650 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.857073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258413-372d-42df-aae1-70f77e690650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.858558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258413-372d-42df-aae1-70f77e690650 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.871289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 116/226] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.872611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258413-372d-42df-aae1-70f77e690650 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.876234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.894218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.895510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.895955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.896510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.898284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.900090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.902823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 111/227] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.904026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.984675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.006680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258413-372d-42df-aae1-70f77e690650 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.040694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29528-d5f4-4474-ab8c-f7d71d31e723 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.054367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.055071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.055417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.056265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.058357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.060211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.063052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 117/228] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.063726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.092681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.094171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.094605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.095340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.097570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.100007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.103513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 112/229] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.104557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.158296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.204653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a661f3-4eb8-4219-b2e6-8ffd601ed7ad None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.232253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7228ef-df90-48d5-a60d-0427fc38183e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.264757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.265349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.265941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.266236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.268029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.269959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.271110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.275105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 118/230] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.276263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.287782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.288411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.288697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.288974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.290537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=6a12010a529b48a4b7c8a47a3648f2a1) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:29.290889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.292429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.301481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 113/231] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/role_assignments?user.id=6a12010a529b48a4b7c8a47a3648f2a1&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=4eed1433d0c54215a270109801b14d73 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.302823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.405349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876f04-bf9b-4cc2-b07a-112a72063979 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.438973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53cf98-6060-4825-a0de-a3ba9d7787d6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.446965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.448067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.448558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.449099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.450761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.452670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.454784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 119/232] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.462312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.481625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.514263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.516259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.516947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.517232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/users/a22e9598b27440c5aac9c2617ad8f86e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.522232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, user_id=a22e9598b27440c5aac9c2617ad8f86e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.524690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd8ad0-8fdc-4891-a671-ccf5630b56f3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.622805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 114/233] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:29 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/users/a22e9598b27440c5aac9c2617ad8f86e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:29.662370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d022-639f-477f-b90c-19c8db0b0a09 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.740886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.741952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.742372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.742829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/users/a22e9598b27440c5aac9c2617ad8f86e/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.747604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, user_id=a22e9598b27440c5aac9c2617ad8f86e, role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.750107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51f64-eed3-463c-8c69-f023e8c074f2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.872907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 120/234] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:29 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/users/a22e9598b27440c5aac9c2617ad8f86e/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:30.249519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb853-a763-4191-b95b-961740030c29 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.250046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb853-a763-4191-b95b-961740030c29 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.250046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceb853-a763-4191-b95b-961740030c29 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.251501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 115/235] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:30.257185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7115-023d-4bde-a81b-0d938ceaadea None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.257667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7115-023d-4bde-a81b-0d938ceaadea None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.257741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7115-023d-4bde-a81b-0d938ceaadea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.279855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d5de7115-023d-4bde-a81b-0d938ceaadea None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:30.281079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d5de7115-023d-4bde-a81b-0d938ceaadea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:30.374416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:30.384689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 121/236] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:30.548729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e09e-e446-4be3-a38e-e2d20dadd253 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.549068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e09e-e446-4be3-a38e-e2d20dadd253 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.549288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e09e-e446-4be3-a38e-e2d20dadd253 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.550854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 116/237] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:30.557090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.557369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.557646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.584888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:30.586095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:30.734842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 122/238] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:30.845833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e09e-e446-4be3-a38e-e2d20dadd253 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:30.915804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:30.987828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e09e-e446-4be3-a38e-e2d20dadd253 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.044816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.045842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.046492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.047046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.065928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:31.071504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:31.072456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:31.073709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 117/239] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:31.082281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.204490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.267554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f1537-2109-4003-b93a-69e3a0c6258f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.314944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.329996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.330958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.331388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.331803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.332863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:31.333314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.336226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.355344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 123/240] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:31.366971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.488771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ce98-9729-4b44-813c-148a7c6b44a6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.516796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.594449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.595136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.595422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.595696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.596857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.598353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566eaa4-e6e1-4c15-b5de-70f91d303afb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.600012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 118/241] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:31.601423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367c8c-f862-49ed-8f6b-2c6d17f8ab26 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.601706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367c8c-f862-49ed-8f6b-2c6d17f8ab26 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.601986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c367c8c-f862-49ed-8f6b-2c6d17f8ab26 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.603350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 119/242] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:31.608311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c367c8c-f862-49ed-8f6b-2c6d17f8ab26 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.610106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb87f0-feca-4327-ad59-42d333035838 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.707314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.708855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.709708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.711508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] PATH_INFO: `/v3/users/f61a8477aacd45118c2a70f777814cd1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.746283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c367c8c-f862-49ed-8f6b-2c6d17f8ab26 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.752231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] RBAC: Authorizing `identity:get_user(user_id=f61a8477aacd45118c2a70f777814cd1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.756623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312a75-7fa0-4784-b149-80e1a80033bc None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.768132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 124/243] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/users/f61a8477aacd45118c2a70f777814cd1 => generated 297 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:31.768381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.768934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.768934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.798671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.799378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.799662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.799941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.800672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.802981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.812105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:31.817486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:31.892530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 120/244] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:31 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:31.893745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.907967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.945308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.963667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 125/245] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:32.030197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5dfc6-a16d-4fc4-b61c-9005ee3c24f8 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.073236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.074268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.074779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.075283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.156701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.158671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0cecd-026a-4061-b9f3-097101235956 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.197995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.275623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 121/246] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:32.284753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.302688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d48df-615f-4840-8e61-c66ee3f33219 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.303003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d48df-615f-4840-8e61-c66ee3f33219 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.303285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d48df-615f-4840-8e61-c66ee3f33219 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.306406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 122/247] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:32.310569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.310923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.311165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.339100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:32.340319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:32.427178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f14b-362d-4ae6-b131-672d6e903ce9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.444257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 123/248] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:32.487108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.496891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.498146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.500031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.500031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.501426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.504610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.510682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 126/249] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:32.515745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.587636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.639923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5658-9104-453c-a659-824bb2b0299c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.647631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.686328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.687319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.687808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.688259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.705031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:32.710221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:32.711293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:32.712676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:32.715931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 124/250] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:32.723772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.775733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae34d-85a7-4617-a689-573422e213e0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.833803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.835943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.835943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.835943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.837821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=f61a8477aacd45118c2a70f777814cd1) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:32.837991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.840636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5c3b8-b3ea-4823-80bd-f9ff8bf20814 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.851885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 127/251] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/role_assignments?user.id=f61a8477aacd45118c2a70f777814cd1&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:32.853221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ead2-f0d5-4e89-a68b-b4184bfdb10e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.853508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ead2-f0d5-4e89-a68b-b4184bfdb10e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.853775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ead2-f0d5-4e89-a68b-b4184bfdb10e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.857890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8f30c-a523-4258-8038-c6bacfad2ccd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.862135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 128/252] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:32.862135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231ead2-f0d5-4e89-a68b-b4184bfdb10e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.936673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.996257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.996915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.997201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.997482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.998198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:32.998508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.000060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29866148-6d62-4916-bd86-931e0766eb27 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.026101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 125/253] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.027204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.027491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.027763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.048312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:33.049546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:33.103462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231ead2-f0d5-4e89-a68b-b4184bfdb10e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.125865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.182339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.183503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.183973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.184870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.208629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:33.219783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:33.221532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:33.223091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 129/254] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:33.228264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.252666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 126/255] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:33.253548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.360842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb3b34-bfe0-4e68-816e-79bb97da0975 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.383334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.418857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.419563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.420182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.420472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.421178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:33.421465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.425369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.447138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 127/256] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.456189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.562341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0fa-11c6-492e-9c63-8826ecfcb18f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.650643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.651241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.651579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.651862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] PATH_INFO: `/v3/users/6a12010a529b48a4b7c8a47a3648f2a1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.659610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] RBAC: Authorizing `identity:get_user(user_id=6a12010a529b48a4b7c8a47a3648f2a1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.661845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.663788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 130/257] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/users/6a12010a529b48a4b7c8a47a3648f2a1 => generated 297 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.674536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.741121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b7d3-8ae7-4971-866e-197e878efd44 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.773284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.860759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.861800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.862283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.862790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.867988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.870830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.874059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 128/258] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.883656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.884824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0be56-4ed8-4525-8503-456cc0896647 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.950779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.951671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.951952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.952225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.957014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.958628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.960545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 131/259] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.961149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.004833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.096181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f5a1e-7405-4629-bff4-20cd9e0c8ffb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.143822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.144852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.145141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.145421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.234370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6082cd-c9b0-40dc-b813-abe61a82c5e7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.245502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.248275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.249796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.356373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.357418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.357904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.358394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.371030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:34.376484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:34.377439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:34.378232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:34.381152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 132/260] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:34.381522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.388390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.511939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858712d3-7655-4b86-8ffe-823ae196cff0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.557085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.557782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.558063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.558584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/users/6a12010a529b48a4b7c8a47a3648f2a1/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.565956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, user_id=6a12010a529b48a4b7c8a47a3648f2a1, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.567498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.587121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 129/261] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:34.588134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.688016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 133/262] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:34 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/users/6a12010a529b48a4b7c8a47a3648f2a1/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:34.699339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.717206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d5cf-fdc0-4d5d-9b30-7d248ae695d4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.754947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.769127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.771248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.771898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.774830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.775894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:34.776383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.778068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.801147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 130/263] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:34.810254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.837585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.878010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008235a-a221-48eb-9040-e06ba8b03ae5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.950289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.950985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.951420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.951549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.955525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.957032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.959247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 134/264] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:34.960497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935df3d-0787-4bc7-a55e-278bd25dffa8 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.982418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.008631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.009677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.010175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.010727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] PATH_INFO: `/v3/users/a22e9598b27440c5aac9c2617ad8f86e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.012483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] RBAC: Authorizing `identity:get_user(user_id=a22e9598b27440c5aac9c2617ad8f86e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.014357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.016702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 131/265] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/users/a22e9598b27440c5aac9c2617ad8f86e => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:35.022696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.114064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.115469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.122612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bb638-ff29-41f3-9498-22a3c16b27b7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.141869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fab38c-405c-433f-b97b-71a560e81a2c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.179573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.180860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.181352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.181813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.182441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.183045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=a22e9598b27440c5aac9c2617ad8f86e) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:35.183157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.183444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.183647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.183743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.186419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.199724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 135/266] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/role_assignments?user.id=a22e9598b27440c5aac9c2617ad8f86e&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:35.307739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.310617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049ece0-c505-45ea-bc62-f986c70fa913 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.448244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.479125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 132/267] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:35.485890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.564666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181fc5-3552-42cf-8204-51a5cf1b6a51 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.564940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181fc5-3552-42cf-8204-51a5cf1b6a51 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.565267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181fc5-3552-42cf-8204-51a5cf1b6a51 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.566535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 133/268] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:35.572837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59075-38f4-4ae3-a4c8-8578508277ed None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.573219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59075-38f4-4ae3-a4c8-8578508277ed None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.573396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59075-38f4-4ae3-a4c8-8578508277ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.607545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-34f59075-38f4-4ae3-a4c8-8578508277ed None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:35.608889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-34f59075-38f4-4ae3-a4c8-8578508277ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:35.669010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587725cc-e034-4d56-bbf4-b7c6aa5f24e9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.739139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.740259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.740741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.741207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.742794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.744514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dca92-69c0-415b-90ec-d6b233dd49b4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.750861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 136/269] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:35.753180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332980f-b560-4afb-831f-d9831ceeceaa None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.753689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332980f-b560-4afb-831f-d9831ceeceaa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.754156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332980f-b560-4afb-831f-d9831ceeceaa None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.756221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 137/270] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:35.759632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332980f-b560-4afb-831f-d9831ceeceaa None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.764180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 134/271] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:35.765851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3b62-e553-4a76-80e6-2dd5fdca19a5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.766305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3b62-e553-4a76-80e6-2dd5fdca19a5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.766643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3b62-e553-4a76-80e6-2dd5fdca19a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.791337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.794543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-175c3b62-e553-4a76-80e6-2dd5fdca19a5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:35.795928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-175c3b62-e553-4a76-80e6-2dd5fdca19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:35.907909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332980f-b560-4afb-831f-d9831ceeceaa None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.935462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 135/272] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:35.938267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee05f11-b35f-4365-b492-6100c1ba0818 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.938767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee05f11-b35f-4365-b492-6100c1ba0818 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.939316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee05f11-b35f-4365-b492-6100c1ba0818 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.941008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 136/273] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:35.949568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.949568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.949568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.006533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:36.009110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:36.013627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.014724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.015097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.015371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.016193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=a22e9598b27440c5aac9c2617ad8f86e) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.016591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.018460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.035145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 138/274] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/role_assignments?user.id=a22e9598b27440c5aac9c2617ad8f86e&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.066503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.071472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.166923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 137/275] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:36.263304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.327467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.363994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216153fe-72c8-48af-92ff-56f08edde7dc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.436255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.437055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.437414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.437868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.464320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:36.475874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:36.477365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:36.478177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:36.479403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 139/276] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:36.486778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.553948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975f9bd-5665-4078-a89b-eaec6106a0b6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.606288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.607279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.607561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.607860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.611933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.614777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 138/277] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:36.620778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.653822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12899ab-24ab-48b6-ad96-c9e60327515a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.689414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.712982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.713934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.714349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.714608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.715582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.716143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.717587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.732312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 140/278] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.733983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.751988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.812066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c650e3-3c0c-4734-9d9d-4cbfcfc9d32e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.863102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.863747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.864031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.864313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.876389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:36.881194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:36.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 139/279] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:36.883405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.907428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3637b2-6f79-4d76-9456-e1f1d2edd4e0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.969602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.970139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.970419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.970730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.971406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.972871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.997304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.005008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867febfd-5ae5-4f66-aca6-754faea3e6ba None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.042069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.042588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.042913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.043194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] PATH_INFO: `/v3/users/f61a8477aacd45118c2a70f777814cd1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.044420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f61a8477aacd45118c2a70f777814cd1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.045941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.047583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 140/280] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/users/f61a8477aacd45118c2a70f777814cd1 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.048371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.108294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 141/281] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:36 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:37.109920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.242808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.249177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedacbf-2fa4-499d-9888-bcb0d97d40d2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.314146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5140-f6bc-4a06-90e1-e2c5df40673d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.317499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.318478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.318975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.319460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.320917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:37.321428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.323938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.340176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 141/282] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.350837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.376531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.377367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.377879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.378227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.380169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.383377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.386382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 142/283] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.395622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.397740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.528285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e14a52-9c37-4f34-917c-f7fe34c16b06 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.573363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.574018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.574297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.574572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] PATH_INFO: `/v3/users/5daf7451290b44bebb9684923017e303` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.591965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] RBAC: Authorizing `identity:get_user(user_id=5daf7451290b44bebb9684923017e303)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.593752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.595537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 142/284] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/users/5daf7451290b44bebb9684923017e303 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.609502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.613722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49499018-216b-4d86-ba08-a1595d7e56fb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.670100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.671363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.671826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.672292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/users/f61a8477aacd45118c2a70f777814cd1/roles/e84585ece55148caad35c7fe5dca9d5f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.675194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, user_id=f61a8477aacd45118c2a70f777814cd1, role_id=e84585ece55148caad35c7fe5dca9d5f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.678811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.687416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.732708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.755732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b8599-647f-464f-b511-5d5386ed9394 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.797526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.797526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.797879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.798334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.799334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, user.id=5daf7451290b44bebb9684923017e303, scope.system=all) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:37.799334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.800694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026321-9c74-466c-a7c4-2b021bd76c75 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.809375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 143/285] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/role_assignments?user.id=5daf7451290b44bebb9684923017e303&scope.system=all&role.id=4eed1433d0c54215a270109801b14d73 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.810768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942319-1157-4dac-a0d3-9cdd18fd5107 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.811099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942319-1157-4dac-a0d3-9cdd18fd5107 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.811392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c942319-1157-4dac-a0d3-9cdd18fd5107 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.812718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 144/286] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:37.815282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 143/287] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:37 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/users/f61a8477aacd45118c2a70f777814cd1/roles/e84585ece55148caad35c7fe5dca9d5f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:37.822852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.822852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.822852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.846439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:37.848101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:37.939997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 145/288] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:38.352841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.575621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc550-28cc-481f-8252-a6f6371bdb33 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.638582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.639623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.640098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.640555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.661042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:38.672198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:38.673942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:38.676039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 144/289] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:38.685077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.862328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.920744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2308-7778-49b3-b592-35c7f3ec590a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.990244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.991291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.991742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.992718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.993818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.994345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.997077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.013397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 146/290] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:39.020408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.149342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.235529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331159-0530-4719-8ab5-2c85a49a75eb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.331217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.332393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.332913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.333426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] PATH_INFO: `/v3/users/6a12010a529b48a4b7c8a47a3648f2a1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.335894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] RBAC: Authorizing `identity:get_user(user_id=6a12010a529b48a4b7c8a47a3648f2a1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.339007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.343255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 145/291] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/users/6a12010a529b48a4b7c8a47a3648f2a1 => generated 297 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:39.352721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.539440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddae96-ada4-4f45-a2f4-7bac5c233187 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.599225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.599857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.600140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.600406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.612384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.706591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.708111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.854257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 147/292] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:39.863026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.112161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c30f-3f6e-4f31-b1ab-440ef6a06c1a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.175850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.201689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.203511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.204358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.205184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.207751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.211004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0994e-c579-41a3-af86-3981407cb30b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.215401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 146/293] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.221829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.301524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.395923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d9e8-2ca4-473d-96b5-c89fedf4e220 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.428692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856dbab-b004-4269-be3f-3d166aafb46e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.429634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856dbab-b004-4269-be3f-3d166aafb46e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.430357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856dbab-b004-4269-be3f-3d166aafb46e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.433408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 147/294] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:40.441360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc668-8a25-4cd5-91e3-0fa97169fb27 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.442291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc668-8a25-4cd5-91e3-0fa97169fb27 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.442944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc668-8a25-4cd5-91e3-0fa97169fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.448822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.449447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.449722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.449991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.450800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=6a12010a529b48a4b7c8a47a3648f2a1) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:40.451112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.452612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63396815-c5e1-4169-8dc7-124ac0c97a5a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.460276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 148/295] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/role_assignments?user.id=6a12010a529b48a4b7c8a47a3648f2a1&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=4eed1433d0c54215a270109801b14d73 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.462277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5158c-1635-4893-baab-9c4295621da0 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.462619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5158c-1635-4893-baab-9c4295621da0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.467184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5158c-1635-4893-baab-9c4295621da0 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.469299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 149/296] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:40.475774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d20e5-c7a4-4fc8-8902-124e69d87286 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.476040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d20e5-c7a4-4fc8-8902-124e69d87286 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.476909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d20e5-c7a4-4fc8-8902-124e69d87286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.480663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-df3dc668-8a25-4cd5-91e3-0fa97169fb27 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:40.482018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-df3dc668-8a25-4cd5-91e3-0fa97169fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:40.511663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ec5d20e5-c7a4-4fc8-8902-124e69d87286 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:40.512964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ec5d20e5-c7a4-4fc8-8902-124e69d87286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:40.636438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 148/297] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:40.685086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 150/298] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:40.815091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1a82-dab2-480f-9701-8033f4c9a091 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.816029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1a82-dab2-480f-9701-8033f4c9a091 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.816816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1a82-dab2-480f-9701-8033f4c9a091 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.820612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 149/299] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:40.827435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.828364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.829149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.861458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:40.863927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:40.923246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.938489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8df2-d94e-4d24-af62-148dababb2eb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.938489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8df2-d94e-4d24-af62-148dababb2eb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.938489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8df2-d94e-4d24-af62-148dababb2eb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.942739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 150/300] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:40.949196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.949705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.950204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.978707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:40.980048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:41.025894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 151/301] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:41.026800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.099979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 151/302] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:41.140559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.208592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.223204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac270c7-a3a6-4e53-85ad-5231c36d3105 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.266526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06b19-f420-4cd2-b279-e841d6adb56f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.293791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132da48-ae62-423e-a394-482185914bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.295572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132da48-ae62-423e-a394-482185914bfa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.296429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132da48-ae62-423e-a394-482185914bfa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.297285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132da48-ae62-423e-a394-482185914bfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.306000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.306640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.306947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.307220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.308410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.309900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.311575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 152/303] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:41.312418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.318952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132da48-ae62-423e-a394-482185914bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-1132da48-ae62-423e-a394-482185914bfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:41.329410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:41.330557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:41.331508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 152/304] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:41.333179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132da48-ae62-423e-a394-482185914bfa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.382146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.465494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f035dc-7e8d-415c-8cbd-851955ad4b6b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.490399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132da48-ae62-423e-a394-482185914bfa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.493267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.512429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.513090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.513369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.513650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.528891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:41.535065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:41.536884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:41.538478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 153/305] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:41.543764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.548248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.549074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.549418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.549834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] PATH_INFO: `/v3/groups` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.550981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.552629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.641756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 153/306] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Aug 24 08:34:41 2022] POST /identity/v3/groups => generated 229 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:41.643747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.661625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a7bb-0104-4c21-9d48-9a558fca5061 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.707805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.708993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.709560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.709937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.711536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:41.712064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.715118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.726596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 154/307] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:41.727966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.772758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.797154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222844-b284-4887-840e-9c2aa62cff02 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.801420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.851402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.852422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.852879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.853327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.875178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:41.888253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:41.889954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:41.891534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 154/308] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:41.892844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.901690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2cad6-1986-48fa-a109-5e53b40ccc84 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.947406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.948341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.948773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.949175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.950238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:41.950722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.952551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.966734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 155/309] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:41.966734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.078417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0484b2-d13e-47c4-8610-1f36efa6c969 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.133265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5221-d374-4382-bfca-e0846a5d7b3b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.135549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.136229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.136406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.137185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.137185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] PATH_INFO: `/v3/users/7d21eada81eb41f4a9e54d06fa564cdb` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.157018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.161030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] RBAC: Authorizing `identity:get_user(user_id=7d21eada81eb41f4a9e54d06fa564cdb)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.163487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.166789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 155/310] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/users/7d21eada81eb41f4a9e54d06fa564cdb => generated 293 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.168054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.207705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.208746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.209245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.209727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.211021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:42.211556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.214301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.236391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 156/311] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.236391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.321697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b500d-3f68-4f09-9c7e-1af4b63d37ce None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.380518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.381602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.382128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.382604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] PATH_INFO: `/v3/users/5daf7451290b44bebb9684923017e303` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.386105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] RBAC: Authorizing `identity:get_user(user_id=5daf7451290b44bebb9684923017e303)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.390425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.399299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608db1f-3f4b-46d2-96be-bd60b4a833f7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.403975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 156/312] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/users/5daf7451290b44bebb9684923017e303 => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.405345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.450358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.451525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.452001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.452458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.453813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, user.id=7d21eada81eb41f4a9e54d06fa564cdb, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:42.454378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.456863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.465669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 157/313] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/role_assignments?user.id=7d21eada81eb41f4a9e54d06fa564cdb&scope.system=all&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.466850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.523633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.559287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.603906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175315b-9c26-4f69-8122-5f00ccd7cb9f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.655811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.657172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.657684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.658181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] PATH_INFO: `/v3/users/f61a8477aacd45118c2a70f777814cd1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.660826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] RBAC: Authorizing `identity:get_user(user_id=f61a8477aacd45118c2a70f777814cd1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.662738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.666275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 157/314] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/users/f61a8477aacd45118c2a70f777814cd1 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.672431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.681388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810224ae-a631-4851-9dfe-1e9c255fcf49 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.756156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.757645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.759031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.759578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] PATH_INFO: `/v3/system/users/5daf7451290b44bebb9684923017e303/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.761635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5daf7451290b44bebb9684923017e303, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.765729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.829771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141217-727d-4062-acbb-e53bf560b17d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.868886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.869503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.869785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.870061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.874158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.880020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 158/315] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Aug 24 08:34:42 2022] PUT /identity/v3/system/users/5daf7451290b44bebb9684923017e303/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:42.963913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.965570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.093208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 158/316] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:43.099282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.216741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.226105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae63ff-8959-4dbe-8225-c069aa2f4f65 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.273888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.275078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.275564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.276349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.278567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.280107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbebf4-bdd5-4a09-8e11-bf14e686dfaf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.282215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 159/317] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:43.288835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.328810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.404077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fea3fb-986c-48e0-a497-a5633310e6c3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.442627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.443280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.443563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.443838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.445127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, scope.project.id=c61ba39450ed482e998b9104e2193aac, user.id=f61a8477aacd45118c2a70f777814cd1) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:43.445444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.446939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c148c-c905-41e6-ab91-1e1a0f2af3ed None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.453838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 159/318] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/role_assignments?user.id=f61a8477aacd45118c2a70f777814cd1&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:43.646959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2051e-880f-4c65-a0d9-0e0523bb766e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.652405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2051e-880f-4c65-a0d9-0e0523bb766e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.652405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2051e-880f-4c65-a0d9-0e0523bb766e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.652405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 160/319] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:43.655539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.655837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.656120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.680082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:43.681303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:43.756317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 160/320] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:44.028372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2051e-880f-4c65-a0d9-0e0523bb766e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.214312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2051e-880f-4c65-a0d9-0e0523bb766e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.280364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.282719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.283445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.284172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.286462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.289922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3940d0-5b18-4cfb-ac74-191dd188633d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.299088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 161/321] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:44.303347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.362130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.489458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047b103-abbb-4859-83ab-9b35f093a279 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.563491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.564463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.564974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.565271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] PATH_INFO: `/v3/groups` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.566447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.569301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.684895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ada25-66c5-4021-938b-1969fe14f08f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.685224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ada25-66c5-4021-938b-1969fe14f08f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.685568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ada25-66c5-4021-938b-1969fe14f08f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.687118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 162/322] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:44.688941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 161/323] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Aug 24 08:34:44 2022] POST /identity/v3/groups => generated 226 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:44.692944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246649e-71d8-4fe9-9c82-9b88a394e070 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.693241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246649e-71d8-4fe9-9c82-9b88a394e070 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.693512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246649e-71d8-4fe9-9c82-9b88a394e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.714578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0246649e-71d8-4fe9-9c82-9b88a394e070 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:44.716437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0246649e-71d8-4fe9-9c82-9b88a394e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:44.746971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.804690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 163/324] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:45.072925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.201005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.219605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a44819-1885-4fe6-a7a1-6ce3e11b8052 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.281788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.283012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.283433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.283906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.306228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:45.316996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:45.317979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:45.318827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:45.321106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 162/325] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:45.328879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c3bf-7123-471d-b085-c8311f4369f1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.329405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c3bf-7123-471d-b085-c8311f4369f1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.329405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c3bf-7123-471d-b085-c8311f4369f1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.331057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 164/326] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:45.331460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.337018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.337320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.337703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.367717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:45.370260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:45.488279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae54af-6f5c-4ad9-bc49-80fa3ca7991e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.530029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.530917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.531444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.531821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.532659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:45.533105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.535180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.542149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 165/327] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:45.555085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 163/328] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:45.558098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.662864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.680005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8f693-125c-4be9-bf81-eb45f8934295 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.727040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.727701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.727981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.728251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] PATH_INFO: `/v3/users/7d21eada81eb41f4a9e54d06fa564cdb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.729482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] RBAC: Authorizing `identity:get_user(user_id=7d21eada81eb41f4a9e54d06fa564cdb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.730980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.732957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 166/329] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/users/7d21eada81eb41f4a9e54d06fa564cdb => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:45.738480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.792418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.946393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.999338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8404-b296-4ab4-9246-dc2cee6c8e24 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.068821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.070411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.070849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.071244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] PATH_INFO: `/v3/system/users/7d21eada81eb41f4a9e54d06fa564cdb/roles/e84585ece55148caad35c7fe5dca9d5f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.074040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7d21eada81eb41f4a9e54d06fa564cdb, role_id=e84585ece55148caad35c7fe5dca9d5f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.077112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.128856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8ea6-187a-4843-9c5a-3a01993bd4a8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.162327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 164/330] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Aug 24 08:34:45 2022] PUT /identity/v3/system/users/7d21eada81eb41f4a9e54d06fa564cdb/roles/e84585ece55148caad35c7fe5dca9d5f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:34:46.173550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d945645-d88f-4746-912f-69accb231bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.174211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d945645-d88f-4746-912f-69accb231bde None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.174485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d945645-d88f-4746-912f-69accb231bde None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.174784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d945645-d88f-4746-912f-69accb231bde None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.197813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d945645-d88f-4746-912f-69accb231bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2d945645-d88f-4746-912f-69accb231bde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:46.202710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:46.203646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:46.204482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 167/331] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:46.214598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.315872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.346165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61c58-498b-4863-a6e8-b6aa799c7ee7 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.396150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.397148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.399437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.399988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.400910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:46.401426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.403080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.415628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 165/332] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:46.423705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d945645-d88f-4746-912f-69accb231bde None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.461376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.597634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d945645-d88f-4746-912f-69accb231bde None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.647004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.647713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.648062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.648351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] PATH_INFO: `/v3/users/5daf7451290b44bebb9684923017e303` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.649688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] RBAC: Authorizing `identity:get_user(user_id=5daf7451290b44bebb9684923017e303)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.651215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437b1b-47eb-479b-b3f8-3b03898ea90d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.653294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 168/333] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/users/5daf7451290b44bebb9684923017e303 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:46.662003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.861369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ee35-bfaa-4127-91db-797870140ab4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.940357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.941424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.941921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.942420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.943869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] RBAC: Adding query filter params (role.id=4eed1433d0c54215a270109801b14d73, user.id=5daf7451290b44bebb9684923017e303, scope.system=all) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:46.944465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.947310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7da7c-a2fd-4542-8ade-41db1a3e1cab None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.961308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.983452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 166/334] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/role_assignments?user.id=5daf7451290b44bebb9684923017e303&scope.system=all&role.id=4eed1433d0c54215a270109801b14d73 => generated 510 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:47.932632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ac78-f454-4c0f-b87c-e80c86ab3839 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.933165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ac78-f454-4c0f-b87c-e80c86ab3839 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.933165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ac78-f454-4c0f-b87c-e80c86ab3839 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.934975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 169/335] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:47.941026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429d27-4722-4cc1-ac8d-cf57397c407e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.941982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429d27-4722-4cc1-ac8d-cf57397c407e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.942501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429d27-4722-4cc1-ac8d-cf57397c407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.968046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c4429d27-4722-4cc1-ac8d-cf57397c407e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:47.969657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c4429d27-4722-4cc1-ac8d-cf57397c407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:48.053056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 167/336] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:48.777623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6ac78-f454-4c0f-b87c-e80c86ab3839 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.817752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1602d-1779-4411-9160-14026cdc5687 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.818033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1602d-1779-4411-9160-14026cdc5687 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.818575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1602d-1779-4411-9160-14026cdc5687 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.820205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 168/337] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:48.826401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.827416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.827911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.851030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:48.852810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:48.934788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 169/338] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:48.953481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.984147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6ac78-f454-4c0f-b87c-e80c86ab3839 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.037516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.038688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.038975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.039246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.059763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a2b390f3-bb31-418b-8ac6-c0559b201cfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:49.064833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:49.065767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:49.066570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 170/339] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:49.074089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.079267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4119-bbff-4ada-b00f-dccb5c3489b6 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.079578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4119-bbff-4ada-b00f-dccb5c3489b6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.079833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4119-bbff-4ada-b00f-dccb5c3489b6 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.081385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 171/340] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:49.087954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.088236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.088513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.127845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:49.129096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:49.212662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.289628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 172/341] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:49.311346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acfc59-be14-4924-ac15-731f7a39cb25 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.363233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.363868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.364327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.364606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.365298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:49.365598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.367075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.384448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 170/342] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:49.390135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.410684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.595139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.654524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3932a-c0b1-47f5-b547-76e488b93e02 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.712776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a997-1ce3-4ffa-8977-1b8492c2cb00 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.723759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.724574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.725384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.726165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.738550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.774559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.775324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.776526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.776990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.796520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-d0e042da-7756-4ea4-8f3a-13b752d1d602 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:49.802238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:49.807582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:49.808568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:49.810976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 171/343] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:49.812480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0330-41bb-49c3-9f02-e844a5ad87ea None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.812924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0330-41bb-49c3-9f02-e844a5ad87ea None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.813336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de0330-41bb-49c3-9f02-e844a5ad87ea None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.815078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 172/344] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:49.821068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff5e16-2154-4c9a-9d9f-fed45a865daf None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.821781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff5e16-2154-4c9a-9d9f-fed45a865daf None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.822202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff5e16-2154-4c9a-9d9f-fed45a865daf None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.824143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 173/345] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:49.825695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.826166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.826641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.847961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:49.849100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:49.870268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.871832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a9267-6564-48bd-88d0-6baf823e2c0d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.959696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 174/346] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:49.961007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.985839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.015446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 173/347] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:50.017455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.018039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.018313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.040763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:50.042211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:50.078584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b636c7-0b80-4e4f-a525-e0ca4100dbc2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.090696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.120695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.121588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.122026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.122493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.123412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:50.123958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.125582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.129748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 174/348] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:50.136073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 175/349] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:50.137206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.137783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.262094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca772b-ff99-4da9-a324-307d27eb300d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.307437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.308452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.308944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.309423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.315505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.317265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.319602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 176/350] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:50.320693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.389228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.401156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e640ed-e538-48ca-b659-bbe8a2530153 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.406075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.487496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.488539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.489047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.489539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.500706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7dee9-aa26-4fb8-96c9-6c8f5cccf189 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.503579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:50.512529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:50.513644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:50.514499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:50.517006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 175/351] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:50.518034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.582480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.583219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.583502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.584110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.665265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b770f-4621-4ce4-ad6d-9e2d012e0731 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.677188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.679469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373486-e903-451a-b8ce-6c1d4d5da0a9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.707706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.708501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.708794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.711051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.718129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.719666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.721594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 176/352] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:50.722303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.731941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.785144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.828850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 177/353] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:50.831385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cdb8a-6d2b-4b74-b644-5d97f69aa39e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.831877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cdb8a-6d2b-4b74-b644-5d97f69aa39e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.832347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cdb8a-6d2b-4b74-b644-5d97f69aa39e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.835876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 178/354] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:50.837518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cdb8a-6d2b-4b74-b644-5d97f69aa39e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.906244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1596f9-f937-49d8-a862-74ced062be7d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.983104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.984250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.984917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.985161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.006797 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:51.017576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cdb8a-6d2b-4b74-b644-5d97f69aa39e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:51.020591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:51.022510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:51.024153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 177/355] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:51.025303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.075617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.076578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.077044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.077725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.078985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.079532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.082626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.092726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.097645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 179/356] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.099091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.200671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.226275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed294b2-9126-4ec3-8b06-1ff7540c1520 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.282182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213c641-3449-4cd6-a995-34b8809f1548 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.289604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.290973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.291721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.292358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.313039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:51.322943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:51.323995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:51.324886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 178/357] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:51.327219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.349690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.351498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.352126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.352579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.353944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=5d21b37c9bd744349387927fa9dbae25) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.357831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.358195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7764bdd-907f-4cdc-b636-edc38f661bce None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.368948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 180/358] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=4eed1433d0c54215a270109801b14d73 => generated 260 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.371259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.371732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.372186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.404292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:51.406694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:51.522931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeb11b-5184-4dd8-b45e-87ceefb3b548 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.543304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.562997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 181/359] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:51.570883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.592607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.593633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.594112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.594631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.596337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.598034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.600430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 179/360] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.602181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.641246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.746918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd52dd-b3ca-49bb-b37d-7054e8ef5fa4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.824268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.825907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.826591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.827253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.828831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.829605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.833194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.834900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce18df-e425-475d-adbe-6b9bf3bcc6f2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.852298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 182/361] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.855046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.912948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.914223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.914843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.915416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.980123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.064227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ec45c-fa30-453b-8895-12caabb7cdc4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.089025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.094003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.122714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.123495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.126375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.127438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.128209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.129796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:52.130643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.135334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.153679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 183/362] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:52.155649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.281255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 180/363] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:52.284807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.358748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355fb9-95be-40f6-a2e7-1fb2391a0915 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.389779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.422325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.424481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.425659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.426081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.428587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.432553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.436176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 184/364] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:52.440685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.449231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4da6d-dc98-4e71-aa70-6957fbe64b96 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.474367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.495986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.497243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.498080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.498562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] PATH_INFO: `/v3/users/7d21eada81eb41f4a9e54d06fa564cdb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.502308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] RBAC: Authorizing `identity:get_user(user_id=7d21eada81eb41f4a9e54d06fa564cdb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.504860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.510115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 181/365] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/users/7d21eada81eb41f4a9e54d06fa564cdb => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:52.511585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.622284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f78c9f-cc7e-44bf-ab33-0eaff7b3005b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.677704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.679106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.679526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.679925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.681582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e918-4ddd-4e41-9c39-9656c6ad6abe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.695200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:52.702086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:52.703930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 185/366] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:52.705514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.730811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.731727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.732164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.732565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.770929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.813618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.830954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.832476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.844261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddd17b-959e-4623-a981-722147f79438 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.883943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.884479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.884757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.885029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.886299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.887822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.889504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 186/367] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:52.890363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.007802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 182/368] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:53.008696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.044406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942715-8aab-45c7-970f-09f64bc1db41 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.063829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.095999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.096641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.096935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.097201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.098117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=5d21b37c9bd744349387927fa9dbae25) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.098851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.101300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.108097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 187/369] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.109117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.169381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.202459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af80e-f862-4a67-9e21-f38a91a22fee None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.281647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.282639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.283145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.283615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.284636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] RBAC: Adding query filter params (role.id=e84585ece55148caad35c7fe5dca9d5f, user.id=7d21eada81eb41f4a9e54d06fa564cdb, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.284946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.289090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.305437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 183/370] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/role_assignments?user.id=7d21eada81eb41f4a9e54d06fa564cdb&scope.system=all&role.id=e84585ece55148caad35c7fe5dca9d5f => generated 510 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.306756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.339120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ae07-ff9d-4844-87ba-644cba9f291c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.412943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.413991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.414484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.415015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.415957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.416944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.419856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.436687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 188/371] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.438133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.538742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0750ef-1b64-4778-9843-c2045e83cd11 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.551024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.605397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.606059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.606337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.606615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.607823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.609318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.611000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 184/372] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.611792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.626236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.701769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf18c-ccbe-47e9-8978-fa9ec43f0a1f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.752579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.753440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.753728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.754004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.755313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.757404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.766830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 189/373] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.766830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.801709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6045e-5a21-4440-9f82-37a6dfead3c3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.889353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.890980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cda21-1dc5-4770-9d70-b447bb517d7d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.891466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.892245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.893064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.902243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.946916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.948369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.949117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.949859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.951551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=c61ba39450ed482e998b9104e2193aac) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.952494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.955616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.968379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 190/374] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=4eed1433d0c54215a270109801b14d73 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.971250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.996606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.998406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ef568-53a6-4d85-b347-0dcef472a436 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.017058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.170732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 185/375] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:54.171662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acdc6-6e0e-466d-a1ab-98de737d850a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.172147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acdc6-6e0e-466d-a1ab-98de737d850a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.173551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acdc6-6e0e-466d-a1ab-98de737d850a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.176385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 186/376] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:54.184986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.185758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.186387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.216854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f461-f5dd-4614-bcbc-417e7a930234 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.220602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:54.223458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:54.290010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.290704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.291003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.291340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.293774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.295356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.300194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 191/377] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.300194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.367337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.375260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 187/378] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:54.375870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.452629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.480808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763bf-d4e7-4a5a-8107-f1b24eebc99a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.532580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d01ad-391a-4a04-8c68-6c186b72857e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.574420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.575079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.575358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.575625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.576370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=c61ba39450ed482e998b9104e2193aac) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:54.576678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.578153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.582981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.584111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.584617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.585002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 188/379] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.585211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] PATH_INFO: `/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.590702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] RBAC: Authorizing `identity:get_project(project_id=a6994c6fdbe24b2e98be4db89dbc57c9)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.595613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.599099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 192/380] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9 => generated 344 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.616444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.820879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25735c8-e0ff-4c93-ada5-2712370d9083 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.828799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.899607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.900273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.900550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.900822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] PATH_INFO: `/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.908345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] RBAC: Authorizing `identity:get_group(group_id=fdc673bd02f04757a2cf7d19ff89ca3f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.909856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.911709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 189/381] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f => generated 229 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.922582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.142310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.184863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14f89-5bbb-44b9-96fe-185b042851cd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:55.222717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:55.285483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:55.287554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.288468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.289230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.291069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] RBAC: Adding query filter params (group.id=fdc673bd02f04757a2cf7d19ff89ca3f, role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=a6994c6fdbe24b2e98be4db89dbc57c9) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:55.291948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:55.302150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:55.315254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66498-61ad-4dc8-bc4d-1cc645852704 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:55.316603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 193/382] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/role_assignments?group.id=fdc673bd02f04757a2cf7d19ff89ca3f&scope.project.id=a6994c6fdbe24b2e98be4db89dbc57c9&role.id=3f587a5233884fd6861867344e2c084b => generated 260 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:55.355301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:55.355970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.356249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.356864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.369533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:55.374750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:55.376752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:55.378672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 190/383] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:55.386989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.664324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39ec5-d402-4f39-a065-7b51781763c5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:55.737511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:55.738552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.739086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.739565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.740865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:55.741412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:55.744035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:55.758956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 194/384] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:55.768738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.847082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.015160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34dc-21eb-4b41-a3f4-876780ab02fe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.097447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.099007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.099828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.100571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.103088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.106230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.115274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 191/385] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:56.119479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.174038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.246706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301f87-fdfd-4667-9c35-c6c35b76ed66 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.282985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.283639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.283923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.284210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.285431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.286966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f95d60-8150-49fb-93ea-c56a3c0a6d1a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.288993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 195/386] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:56.298073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.541907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.550928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4944e1-48b9-45ef-af8a-f4f5963139be None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.602994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.603686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.603966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.604239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/groups/5772af09197f45f1b773a806d351a6bc/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.612227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, group_id=5772af09197f45f1b773a806d351a6bc, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.613710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f612629-c982-4983-a3e9-b58cefa074e2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.689541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 192/387] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:56 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/groups/5772af09197f45f1b773a806d351a6bc/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:56.911018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce7b53-e9e4-4ef2-b28e-12207d74d5bd None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.911802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce7b53-e9e4-4ef2-b28e-12207d74d5bd None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.912625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce7b53-e9e4-4ef2-b28e-12207d74d5bd None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.915741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 196/388] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:56.922941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.923276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.923484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.939683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe136-c8d6-4bb4-b04a-0bfe1a6465d8 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.940589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe136-c8d6-4bb4-b04a-0bfe1a6465d8 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.941306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe136-c8d6-4bb4-b04a-0bfe1a6465d8 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.944468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 197/389] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:56.947832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:56.949100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:56.954006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.954919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.955650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.999153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:57.002942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:57.080227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 193/390] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:57.139585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 198/391] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:57.618777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.619674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.671668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.802258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.821262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b1321-9aee-4dea-9990-87d694ee3472 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.829519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e63fc5-689e-4e85-8645-91fa7f98c5a7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.871018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.871954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.872651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.873083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.879251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.880719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.881598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.882168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.885305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3759eff5-927b-433e-9017-49a072fb1244 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:57.891441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:57.892447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:57.893449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 194/392] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:57.894343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9613-3b3e-4d9e-9fca-000fe4428da8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.894534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9613-3b3e-4d9e-9fca-000fe4428da8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.894969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9613-3b3e-4d9e-9fca-000fe4428da8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.901578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 195/393] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:57.901578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a9613-3b3e-4d9e-9fca-000fe4428da8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.903040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-45f8a3e0-4fbf-4052-942e-d10387a608f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:57.908489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:57.909361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:34:57.910539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 199/394] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:57.912527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.912527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.912527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.938677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:57.940488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:58.047038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 200/395] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:58.047908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.070457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a9613-3b3e-4d9e-9fca-000fe4428da8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.100887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.140113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.140764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.141100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.141377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.142087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:58.142391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.143907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.167408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 196/396] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.172775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.204624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.329778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff80c8-7f76-4f25-964d-30b626a5c623 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.400112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.400934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.401327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.401602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.402599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:58.402935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.404386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.431411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 201/397] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.440438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.463605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b18b7-9c45-49b4-b8dd-d3b833a6c80c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.513615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.515125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.515860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.516643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.519240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.522451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.527500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 197/398] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.529711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.621213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.648135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b00765-6f8b-4b59-b0a3-871d2e4e8568 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.673357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.711251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.712270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.712743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.713200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.714220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ccc94-e197-4ae7-8580-72aea1e89e58 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.715148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.717829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.720120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 202/399] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.721479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.784087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.785341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.785828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.787081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.803590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9c3b1c8a-46f6-4b2e-9afa-dae35f0f7723 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:58.812793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:58.814614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:58.816260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 198/400] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:58.817601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f85c-bde4-4b92-8b88-3fa7deb5f6d8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.817601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f85c-bde4-4b92-8b88-3fa7deb5f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.817601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f85c-bde4-4b92-8b88-3fa7deb5f6d8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.820082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 199/401] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:58.821348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f85c-bde4-4b92-8b88-3fa7deb5f6d8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.881310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c2ef-e73d-4634-b628-0e8eef0db64e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.921968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.922631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.922939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.923213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.924390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.925843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.928451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 203/402] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.929257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.959362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.989201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f85c-bde4-4b92-8b88-3fa7deb5f6d8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.045255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.045899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.046172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.046456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.047638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.049085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a19be7-02b1-48a2-b133-3a97a991fc93 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.056297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 200/403] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:59.056297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.056297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.056297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.062492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.073039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda443c-d2fb-40ca-b0da-1ce6fb9d1166 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.076337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:59.077570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:59.119712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.120446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.120730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.121051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.121743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:59.122047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.123528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.138737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 204/404] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:59.144769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.176788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 201/405] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:59.178614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.284189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.315038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791509c8-be65-4be7-b9c8-e412bf1cc693 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.337601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f9a47-18f5-40b7-ab30-1063b7d80a1d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.377090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.378334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.378786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.379202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/groups/5772af09197f45f1b773a806d351a6bc/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.380925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.381597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.381879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.382162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25/groups/5772af09197f45f1b773a806d351a6bc/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.382530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, group_id=5772af09197f45f1b773a806d351a6bc, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.384357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d21b37c9bd744349387927fa9dbae25, group_id=5772af09197f45f1b773a806d351a6bc, role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.385261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead4901-d606-480c-95c9-47cd3a7e76da None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.385780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.390960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.473286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 205/406] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:59 2022] PUT /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25/groups/5772af09197f45f1b773a806d351a6bc/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:59.474766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.499524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 202/407] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:59 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/groups/5772af09197f45f1b773a806d351a6bc/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:59.512226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935d2c-656f-47b1-9a1c-a300540e036e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.512531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935d2c-656f-47b1-9a1c-a300540e036e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.512780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935d2c-656f-47b1-9a1c-a300540e036e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.514271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 203/408] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:34:59 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:59.515168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32935d2c-656f-47b1-9a1c-a300540e036e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.668919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1575d2-fc1a-4fb4-91fd-d46f65cfeb12 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.727992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.728994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.729277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.729554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.730850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.732611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da71de3-93c1-433d-8558-032c9a531c36 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.734899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 206/409] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:59.740891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.741373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.741845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.761400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32935d2c-656f-47b1-9a1c-a300540e036e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.766544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:59.768157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:59.781026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.821810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.822876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.823534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.824014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.837936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:59.844355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:59.845326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:59.846157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:34:59.849092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 204/410] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:59.850362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.912705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 207/411] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:59.914593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.919923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.984308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610cc47-14a6-448c-897c-792310ef115a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.043490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.044142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.044431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.044694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.051618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.053674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.062286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 205/412] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.062286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.146623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b1774-4e37-4424-b952-e020f22c2d01 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.247539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.248077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.248369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.248645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.249339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:00.249643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.251098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.270957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 208/413] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.271228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91802-6f23-458a-8084-f23586ed9d37 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.289802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.303141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.323421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.349253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.350098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.350563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.352050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac/groups/5772af09197f45f1b773a806d351a6bc/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.357252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] RBAC: Authorizing `identity:create_grant(project_id=c61ba39450ed482e998b9104e2193aac, group_id=5772af09197f45f1b773a806d351a6bc, role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.359222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.471976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a970-4d2d-4c96-b846-4ce5c6834a65 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.488451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 206/414] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Aug 24 08:35:00 2022] PUT /identity/v3/projects/c61ba39450ed482e998b9104e2193aac/groups/5772af09197f45f1b773a806d351a6bc/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:35:00.496714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.521065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.521727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.522016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.522297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] PATH_INFO: `/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.523721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] RBAC: Authorizing `identity:get_group(group_id=fdc673bd02f04757a2cf7d19ff89ca3f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.525309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.527394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 209/415] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:00.553496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.717094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.756964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea265e-61ed-4f63-bd2b-cd3d700afb7a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.759708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9b248-1a04-42f1-8ea2-2e8fc4b787ad None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.814491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.815108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.815432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.815707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] PATH_INFO: `/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.816842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] RBAC: Authorizing `identity:get_project(project_id=a6994c6fdbe24b2e98be4db89dbc57c9)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.818366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.820715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 210/416] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:00.832284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.843659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.845567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.847047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.850958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.881481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:00.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:00.898172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:35:00.899722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:00.900828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 207/417] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:00.910768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.970220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.044953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3a3ff-a97d-4e4b-b091-ba6e51358b4c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.083995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.084666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.085021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.085361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] PATH_INFO: `/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9/groups/fdc673bd02f04757a2cf7d19ff89ca3f/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.087530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6994c6fdbe24b2e98be4db89dbc57c9, group_id=fdc673bd02f04757a2cf7d19ff89ca3f, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.089972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.103199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d854c-2121-4c8d-9ef0-12aeae615724 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.141535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.142257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.142534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.142865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.143694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.144004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.145462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.156415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 208/418] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:01.162226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.167964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.174045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 211/419] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Aug 24 08:35:00 2022] PUT /identity/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9/groups/fdc673bd02f04757a2cf7d19ff89ca3f/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:01.303103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5b782-2465-4c3c-9bbf-82f6ebcd2083 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.307875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.370495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.371540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.371988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.372477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.530405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.540429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.762305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 209/420] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:35:01.769127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.001818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b472aa-8107-4725-8980-98f5acb9de97 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.066141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.067147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.068175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.068645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.070683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.073455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c15b7f-2989-4251-ba41-8efe3def84ee None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.076948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 212/421] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:02.085616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.094829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.108567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32877-ad9e-4e20-abbb-d364dede7853 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.109089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32877-ad9e-4e20-abbb-d364dede7853 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.123925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32877-ad9e-4e20-abbb-d364dede7853 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.123925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 213/422] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:02.123925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.123925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.123925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.141627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:02.142907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:02.217897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.226418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 214/423] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:02.321795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223aa9d-c5d5-452d-8245-c57db4389241 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.377847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.378515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.378826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.379108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.383546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.385209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbab32e-6835-46d4-b454-ad62b5b4c074 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.387540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 210/424] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:02.394697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.407746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d97cc-29c9-4986-84bf-da513803b86a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.408176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d97cc-29c9-4986-84bf-da513803b86a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.408340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d97cc-29c9-4986-84bf-da513803b86a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.410059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 211/425] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:02.418750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.419043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.419351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.441965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:02.444020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:02.674529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 212/426] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:02.706807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8cf6b-bf7f-42d1-9e5c-1519ef84e27b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.788050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.789176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.789685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.790171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.791727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=5d21b37c9bd744349387927fa9dbae25) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:02.792296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.795112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.811526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 215/427] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=4eed1433d0c54215a270109801b14d73 => generated 611 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:02.914940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.919476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.138644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94337a1-7b42-4456-8b25-6526c4f0d670 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.198088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.199748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.199748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.199748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.221349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:03.226468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:03.228587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:03.230433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 213/428] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:03.240412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.274947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.370730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.423864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.461012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246ce0-d462-484a-b135-62665458b5aa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.502371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808edd-989a-49f4-88ce-699a8ce4b92d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.546599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.550230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.551118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.552401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.553907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] RBAC: Adding query filter params (name=member) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:03.554775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.559830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.577639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 216/429] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:03.584509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.592348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.592998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.593278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.593536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.606563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:03.615339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:03.617279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:03.619171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 214/430] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:03.629486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.799396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0acf-dd98-49e6-8e33-9b4246dcacff None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.836403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a06f-945a-48f4-b8ba-60cc2c4e6d12 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.838646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.863319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.864661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.865142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.865624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.878980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.905751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.906689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.907153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.907607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.908536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:03.908717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.911689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d861-7e7c-4c28-89cb-8d1ad6d84b08 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.926532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 215/431] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:03.928682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076092df-c9f8-4114-b26e-48487a2121cb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.929183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076092df-c9f8-4114-b26e-48487a2121cb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.929669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076092df-c9f8-4114-b26e-48487a2121cb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.932685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 216/432] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:03.942003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.942556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.943054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.968801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:03.970778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:03.998433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.000854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.084570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 217/433] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:04.092182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.184738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 217/434] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:04.190156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.246818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.271622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086dcb8-b050-4caf-8eb2-0be46eb32dd5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.302092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.328568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.329353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.329734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.330108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.379111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a171aa-f7bd-4afb-af49-2abd03e1da71 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.434537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.435461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.435806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.436167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.436334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.437636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.438049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.439342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd9598-1ac4-4543-9fec-ae6f7c631a73 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.441356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 218/435] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:04.444153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2399772-8c05-4e17-9d9e-451bb6770bac None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.444636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2399772-8c05-4e17-9d9e-451bb6770bac None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.445108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2399772-8c05-4e17-9d9e-451bb6770bac None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.447707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 219/436] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:04.449324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2399772-8c05-4e17-9d9e-451bb6770bac None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.575393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.588333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2399772-8c05-4e17-9d9e-451bb6770bac None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.605913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 218/437] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:04.606993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.624204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.636696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.637237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.637533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.637770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.653308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9fbb58b8-68ad-43fa-b281-1da2f52e2cc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:04.663070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:04.664513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:04.665525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 220/438] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:04.667416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.667920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.668401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.698027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:04.699331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:04.731585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871975d-2017-4048-b4de-e0aeed1bd819 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.770374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 221/439] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:04.771383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.796236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.797631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.798337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.798859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.800929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.804327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.815160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 219/440] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:04.816625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.901367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.933582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5eff6-3403-4f03-9b41-4f209901566f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.942474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.967809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21a55-ce5a-4b48-b688-63c9c76eb510 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.999979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.001049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.001547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.002407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] PATH_INFO: `/v3/projects/5d21b37c9bd744349387927fa9dbae25` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.004048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d21b37c9bd744349387927fa9dbae25)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.006883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.010361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 222/441] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/projects/5d21b37c9bd744349387927fa9dbae25 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.011782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.021304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.022094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.022526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.022985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.023844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.024273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.025859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.042162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 220/442] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.042162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.183458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106c652-173a-4038-af8a-b7bfe4878df1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.188986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94460f-1403-45bd-864b-7ff8d01b2e27 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.227250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.227896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.228178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.228448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.229648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.231183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.232814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 221/443] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.233625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.244028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.244765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.245190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.245578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.246480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=4eed1433d0c54215a270109801b14d73, scope.project.id=c61ba39450ed482e998b9104e2193aac) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.246913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.248793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.250205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.256202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 223/444] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=4eed1433d0c54215a270109801b14d73 => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.257047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.288994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.375144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b17c3-d3b2-46e7-8240-e4e1003f461f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.417268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f349c-afce-4c26-9596-81f9b8de9057 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.420135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.420966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.421423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.422664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.477865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.478786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.479200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.479671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.499674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:05.507509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:05.508482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:05.509306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:05.509955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 224/445] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:05.510762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.550869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.553907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.571684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.639878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.684670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a36-7b8a-440c-b1c0-f2bd0e6611ad None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.701408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 222/446] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:05.702479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.741721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.742547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.743064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.743521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.744776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=5d21b37c9bd744349387927fa9dbae25) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.745283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.747460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.772112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 225/447] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=5d21b37c9bd744349387927fa9dbae25&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.772112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.843362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c105fdd-ffe3-4dab-8d91-a20bc248633b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.861566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.892786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.894048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.894542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.895622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.897210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.897698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.899845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.911913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 223/448] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.921498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.954268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40798f-7b0d-4528-8e69-7e110f39c844 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.007778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.008359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.009229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.009229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] PATH_INFO: `/v3/projects/c61ba39450ed482e998b9104e2193aac` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.010963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] RBAC: Authorizing `identity:get_project(project_id=c61ba39450ed482e998b9104e2193aac)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.012129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.013817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 226/449] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/projects/c61ba39450ed482e998b9104e2193aac => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.019672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.025938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.077350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed364-8c6d-474b-b8c8-46f24ca72672 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.120588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.121219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.121500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.121772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.179678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac331071-9584-4891-a0d8-f03d35697fcb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.188549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.235601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.236574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.237051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.237317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] PATH_INFO: `/v3/groups/5772af09197f45f1b773a806d351a6bc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.237785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.238765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] RBAC: Authorizing `identity:get_group(group_id=5772af09197f45f1b773a806d351a6bc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.239718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.240243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.243888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 227/450] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/groups/5772af09197f45f1b773a806d351a6bc => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:06.248709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.366111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.384617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 224/451] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:35:06.393573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.422601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6dce7-2f5d-4174-8fae-e253227f2805 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.474813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.475501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.475897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.476285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.477340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] RBAC: Adding query filter params (group.id=5772af09197f45f1b773a806d351a6bc, role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=c61ba39450ed482e998b9104e2193aac) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.477842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.479950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.488021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 228/452] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/role_assignments?group.id=5772af09197f45f1b773a806d351a6bc&scope.project.id=c61ba39450ed482e998b9104e2193aac&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.488851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.526349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68a90-1ec5-4225-adcd-26ecd6c3d3ef None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.573145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.573818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.574095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.574373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] PATH_INFO: `/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.575805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=a6994c6fdbe24b2e98be4db89dbc57c9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.577334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.579217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 225/453] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/projects/a6994c6fdbe24b2e98be4db89dbc57c9 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.586132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.730490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebfa35-1941-45f9-9c58-4c6e60b93154 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.776207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.776850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.777131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.777405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] PATH_INFO: `/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.778548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] RBAC: Authorizing `identity:get_group(group_id=fdc673bd02f04757a2cf7d19ff89ca3f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.780027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c372f-f8c6-4732-a0ae-8a772dd383ad None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.781616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 229/454] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/groups/fdc673bd02f04757a2cf7d19ff89ca3f => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.787706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.905547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720477-52e3-4596-bb82-10a8acb31cd9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.940259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.940873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.941153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.941425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.942805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] RBAC: Adding query filter params (group.id=fdc673bd02f04757a2cf7d19ff89ca3f, role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=a6994c6fdbe24b2e98be4db89dbc57c9) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.943114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.944606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d1f71-d112-4951-981b-7af09fbfc876 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.951077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 226/455] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/role_assignments?group.id=fdc673bd02f04757a2cf7d19ff89ca3f&scope.project.id=a6994c6fdbe24b2e98be4db89dbc57c9&role.id=3f587a5233884fd6861867344e2c084b => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:09.293615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc81b04-02ae-43ab-a567-d11eab95e888 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.294713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc81b04-02ae-43ab-a567-d11eab95e888 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.295208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc81b04-02ae-43ab-a567-d11eab95e888 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.297179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 230/456] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:09.305654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d16e-4213-452e-996a-452456beb8f3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.306620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d16e-4213-452e-996a-452456beb8f3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.307142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d16e-4213-452e-996a-452456beb8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.321583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.329641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-63b1d16e-4213-452e-996a-452456beb8f3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:09.331256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-63b1d16e-4213-452e-996a-452456beb8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:09.437059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 227/457] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:09.698677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7032f3-bb59-4722-a955-70af1ba1e322 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.699440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7032f3-bb59-4722-a955-70af1ba1e322 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.699928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7032f3-bb59-4722-a955-70af1ba1e322 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.701949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 231/458] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:09.708741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.709361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.709867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.714559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb3f52-8f7a-4f40-a28a-d849150f71d5 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.715271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb3f52-8f7a-4f40-a28a-d849150f71d5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.716001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb3f52-8f7a-4f40-a28a-d849150f71d5 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.717916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 232/459] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:09.724224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.724524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.724798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.733101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:09.734925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:09.736020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.756024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:09.757635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:09.888176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 228/460] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:09.895398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 233/461] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:09.904521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.061014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.149200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0f0dc-851b-4d59-a743-1fcafda92886 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.200893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.201973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.202456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.203021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.221813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.224267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:10.245018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:10.246025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:10.246974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:10.246974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.246974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:10.248922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 229/462] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:10.253069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.258485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.366968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc134359-b9b2-45be-8f6b-e5ec32f29d5b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.380890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431dab0-c0c9-4fb0-b5f3-d5bd17eb9e4a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.402133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.405926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.406580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.406911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.407191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.409334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.419099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.420585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.424962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.426014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.426526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.427036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:10.438511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:10.439690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:10.440543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:10.440543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.440543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:10.443203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.444628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 234/463] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:10.445277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.445612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:10.470230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:10.471501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:10.472501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:10.472501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:10.472501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:10.475549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 230/464] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:10.476881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.570477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ccd4-5b1e-46f2-b536-6ac2c4ded341 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.603684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b94f7-fb2e-4bfc-b137-c77c477d80b0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.612126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.613366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.613366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.613580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.614157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:10.614464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.615948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.630438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 235/465] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:10.631294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.649282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.651172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.658610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.658610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.658610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:10.658610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.658610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.681943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 231/466] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:10.683507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.691496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.724991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.796656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b837f43-646e-45dc-af2c-88f17e86e121 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.806328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4286f-980f-4bdd-b1cd-c8c42d7326fa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.845322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.846261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.846773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.847243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.848235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.850119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.864081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.865123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.865713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.866188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.867389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:10.867923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.870544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.898848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 236/467] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:10.898848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.945608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 232/468] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:10 2022] POST /identity/v3/users => generated 246 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:10.947177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:11.098138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686a92f-e0a1-423f-8d33-c7d9b8e5b8a8 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:11.107234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:11.139361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:11.140000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.140282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.140557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.141219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:11.142797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20fa73-4c65-4277-ab71-a4ef5d8d7378 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:11.145351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:11.172254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a7b0b-d4b6-4e12-9c61-083ee61184ec None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:11.216073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:11.217111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.217558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.217950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.218683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:11.221135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440f2d3-3148-48f1-af56-7f712f682d9b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:11.258448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 237/469] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:10 2022] POST /identity/v3/users => generated 248 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:11.259857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b92cc-ac12-478b-970d-e8b97113fe1f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.260062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b92cc-ac12-478b-970d-e8b97113fe1f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.260350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b92cc-ac12-478b-970d-e8b97113fe1f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.262029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 238/470] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:11.262964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f872-c440-415d-a44f-19e2820d08e4 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.263249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f872-c440-415d-a44f-19e2820d08e4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.263641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f872-c440-415d-a44f-19e2820d08e4 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.265087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 239/471] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:11.271263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.271862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.272414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.301429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:11.302861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:11.344061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 233/472] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:10 2022] POST /identity/v3/users => generated 248 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:11.345383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.345687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.345933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.380682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:11.385260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:11.425547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 240/473] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:11.495825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 234/474] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:11.780056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.000487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.017651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ba87a-7cba-4429-bc37-b7f530517dcb None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.061941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.074206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.074869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.075155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.075431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.076108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.077502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.131748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.sql.core [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:35:12.131748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:35:12.131748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [parameters: {'id': '3086eb8022c0425ea9b6390132891d3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:35:12.131748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:12.166418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:12.168506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application yield resource Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise self.value Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application session.commit() Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:35:12.171818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:35:12.174062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:12.176818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:12.191066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [parameters: {'id': '3086eb8022c0425ea9b6390132891d3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.192464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:35:12.193298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.194128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 241/475] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Aug 24 08:35:11 2022] POST /identity/v3/roles => generated 144 bytes in 403 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:12.194128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.248097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.314581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dbee2-09d9-44e9-a9df-6e4ba8da9322 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.456742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.457777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.458245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.458741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.494378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccc1d-bac9-48b1-95b7-a02ad6d5a35f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.501313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.504039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:12.560858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:12.562183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:12.563081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:12.563081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:12.563081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:12.571354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 235/476] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 510 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:12.582577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.586150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.587199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.587659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.588110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.633043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:12.651532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:12.654440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:12.655299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 242/477] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:12.663892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.706187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.803534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a007f2-e27f-4287-b91a-4fd60c2d4beb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.834863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.860700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.862862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.863979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.864549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.865493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:12.866010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.872966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.899253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 236/478] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:12.908323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.926437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64c95c-702d-4a32-a4c4-816c2c52318d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.011955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.012992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.013438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.013908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.015105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:13.015653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.018346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf0de8-de06-4a94-87c8-84c8ed714e6e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.048540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 243/479] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:13.078680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d005cf-bb22-4a55-ba8a-ad4d820bdb3e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.127914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.128940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.129425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.129938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.130858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.132543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cd506-a01f-4c14-b9f8-5a6fe17abbe4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.224830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 237/480] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:12 2022] POST /identity/v3/users => generated 249 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:14.178286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e93e0-8327-437d-b795-4c43b5632c63 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.178867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e93e0-8327-437d-b795-4c43b5632c63 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.178867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e93e0-8327-437d-b795-4c43b5632c63 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.180469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 244/481] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:14.186336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad796bc5-6e8f-4956-a041-a9e655ab47b2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.186887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad796bc5-6e8f-4956-a041-a9e655ab47b2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.186887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad796bc5-6e8f-4956-a041-a9e655ab47b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.215025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ad796bc5-6e8f-4956-a041-a9e655ab47b2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:14.216509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ad796bc5-6e8f-4956-a041-a9e655ab47b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:14.231444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca5c-edfd-4e7b-873e-4f389fcd500a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.231753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca5c-edfd-4e7b-873e-4f389fcd500a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.231997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca5c-edfd-4e7b-873e-4f389fcd500a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.233518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 245/482] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:14.244379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef0514-5699-4faf-af9b-f050a5a4cb39 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.244678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef0514-5699-4faf-af9b-f050a5a4cb39 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.244928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef0514-5699-4faf-af9b-f050a5a4cb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.288552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.296742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a1ef0514-5699-4faf-af9b-f050a5a4cb39 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:14.300086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a1ef0514-5699-4faf-af9b-f050a5a4cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:14.360069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 238/483] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:14.441887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 246/484] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:14.552584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8e3d-4e2a-4170-a41b-a436c1bc408d None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.553240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8e3d-4e2a-4170-a41b-a436c1bc408d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.553717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8e3d-4e2a-4170-a41b-a436c1bc408d None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.555559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 239/485] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:14.563634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.564289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.564764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.588391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:14.589965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:14.593041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.695922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf8e3d-4e2a-4170-a41b-a436c1bc408d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.714442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 247/486] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:14.830440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf8e3d-4e2a-4170-a41b-a436c1bc408d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.878455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.879173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.879456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.879718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.897239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:14.902125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:14.904280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:14.906071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 240/487] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:14.912217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.986056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.053116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.126182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2375a-d1ff-472a-ad38-e94f83a96031 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.199967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.204435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.205320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.206124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.210789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:15.211638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.214770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.239084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.255618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdcd-978a-4f75-b04c-c9c9cbd37d5b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.257518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 248/488] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:15.274522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.321361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.322623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.323233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.323737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.336791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:15.342512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:15.343496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:15.344302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:15.347272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 241/489] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:15.348366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.482368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.485249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8084c-65eb-43b8-acc8-1a08244599aa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.495674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471c60a-5d6f-49ce-8c8b-bfc08ce14ef8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.551388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.552189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.552569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.552943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.566472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:15.571673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:15.572746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:15.574953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:15.577101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 242/490] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:15.578810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.579837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.580582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.581412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.613401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.617681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:15.656694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:15.658966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:15.661083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:15.661083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:15.661083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:15.665336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 249/491] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:15.668387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.747756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3f75-88a8-4df5-8c39-29ed94125749 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.753070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.806106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.806972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.807359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.807740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.808544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:15.808948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.810548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.825888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 243/492] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:15.827122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.845165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.849254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599efdb5-1596-4a0a-b277-29a9f4faf094 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.897174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.897985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.898371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.898779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.899642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:15.900051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.901701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.912437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 250/493] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:15.913306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.006911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1ca0c-87cb-42b8-a234-fd7a8513d444 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.047504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.048338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.048715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.049094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.049867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.050950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.052737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.062283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939050-7c7d-4420-bfac-be9013f2412b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.066955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 244/494] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.067949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.083386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.113310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.114259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.114738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.116012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.131832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.134303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.154696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:16.156838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:16.157945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:16.162172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:16.162172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:16.162172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:16.162172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 251/495] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:16.162570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.198944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab2f3-c9f0-4b83-9103-af455f46e757 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.239794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.240636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.241027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.241405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.253879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.255615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:16.272082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:16.273665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:16.274694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:16.274694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:16.274694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:16.275436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 245/496] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:16.276453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.311617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34eb8f-b801-49eb-a7c4-3d35f9db4108 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.354106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.355078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.355510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.356125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.367516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.368982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:16.382836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.383854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:16.384882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 252/497] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:16.385556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.417114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309ac1-d36f-4114-9887-072d8b891829 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.425881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.454150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.467596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.468218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.468602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.468785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.469494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.469883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.471265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ce293-86d0-451e-91fd-8bba7314691f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 246/498] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98174-0fbf-4107-8b93-4248550f26cc None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98174-0fbf-4107-8b93-4248550f26cc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b98174-0fbf-4107-8b93-4248550f26cc None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 247/499] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:16.490702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b98174-0fbf-4107-8b93-4248550f26cc None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.516925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c3d09-7007-4f56-a695-0c40e8b49101 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.557110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.557722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.558006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.558283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.558964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.559272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.560662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4841d5-dfa3-4e0a-9729-6234db4bbc3c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 253/500] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe7f9d-982f-48cf-9c72-be833d7b560e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe7f9d-982f-48cf-9c72-be833d7b560e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe7f9d-982f-48cf-9c72-be833d7b560e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 254/501] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:16.573919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe7f9d-982f-48cf-9c72-be833d7b560e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.610577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b98174-0fbf-4107-8b93-4248550f26cc None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.647561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.648184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.648462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.648737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.651361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.651361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.651361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ff870-5e56-4f8d-a019-4078a4b2c4db None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.675181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.691802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 248/502] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.693223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29479cd-4a61-478c-b328-6c2781e0b795 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.693488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe7f9d-982f-48cf-9c72-be833d7b560e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.693642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29479cd-4a61-478c-b328-6c2781e0b795 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.693799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29479cd-4a61-478c-b328-6c2781e0b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.713842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f29479cd-4a61-478c-b328-6c2781e0b795 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:16.715387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f29479cd-4a61-478c-b328-6c2781e0b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:16.735067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.736071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.736370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.736660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.746045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.747606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.756428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:16.760252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.761995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 255/503] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:16.763907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.798746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 249/504] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:16.799913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.800299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.800561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.824257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:16.825484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:16.885985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68c15f-2233-46cb-bfe5-b95c2045f53c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.914719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 250/505] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:16.914719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.943642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.943642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.943642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.943642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.965403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.969895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:16.997865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.999914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:17.001871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:17.007413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 256/506] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:17.010409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.030871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.059805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b8a6a-9791-4962-a04e-08fe148c2f2d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.061819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.102298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.102970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.103249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.103638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.183426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.185058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.199605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144134a-d206-4e1a-ae97-a7f676833e02 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.242720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.243354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.243636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.244038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.244658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:17.244986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.246467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.290461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 257/507] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:17.291432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.315139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 251/508] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:17.315139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.316715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.390344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.434144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c66f-6272-4316-bfa5-97857d038d88 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.444009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d12a47-9fa0-41e7-a08e-74b37a689e7d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.472922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.473664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.474015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.474355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.487719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:17.489931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.490689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.490969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.491238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.491993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:17.492298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:17.493599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:17.495422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:17.497058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:17.498239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.499894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 252/509] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:17.500996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.516981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 258/510] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:17.517840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.585779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.627627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464e2f3-ec29-4022-aa54-5c2c1f1e8ea7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.635199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ea8c6-6aaa-40fa-94b8-d0325e1c588f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.660335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.664704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.665309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.665589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.665856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.672667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.673362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.673676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.675221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.685177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.687055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:17.704675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:17.705802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:17.712056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:17.712056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:17.712056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:17.712056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 259/511] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:17.712056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.747472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.748990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.821791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.825528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eea76-9ee7-47ea-94f3-71184c63325d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.863127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 253/512] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:17.863721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.863904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.864621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.865033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.865449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.879478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.880969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:17.896892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:17.898886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:17.900420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:17.900420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:17.900420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:17.900420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 260/513] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:17.903582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.932137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.969878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46df2-3aac-4286-a84c-ae7addacf446 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.007841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.008663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.009075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.009486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.010255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.010645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.011064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc11ec0-64ee-4f21-b724-f0244454bef2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.012332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.021802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 254/514] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:18.022444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.053972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.054613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.054911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.055198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.055601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.133464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.133731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f8ce-c2ce-47a3-932a-ca7e48d7c091 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.135291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.163525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.168534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.169151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.169418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.169782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.170381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.170703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.176890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.185382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 255/515] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:18.186238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.249062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 261/516] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:18.250019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.273233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.290620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30359dd7-4952-46f1-9b32-599d3adfd3a9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.328989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.329650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.329885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.330155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.340711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.342172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:18.357888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:18.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:18.360876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:18.360876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.360876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:18.360876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 256/517] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:18.361648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead58ab7-aab2-4481-bbd5-c27c0e7ea3eb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.365041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.389906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.400381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.401143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.402680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.402680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.402680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.402680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.403902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.416446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 262/518] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:18.417314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.476586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7bf2a-ad6d-4c86-bd3d-4b7b4bea98da None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.515163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.515798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.516078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.516342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.522454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.530857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.538085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.538315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aed12-5c58-4451-af37-e1aea748c084 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:18.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:18.556200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:18.558511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:18.558511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.558511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:18.558511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 257/519] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:18.558511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.578144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.579116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.579400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.579672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.587144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.588700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:18.602491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:18.603841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 263/520] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:18.605936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.656500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.673288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3b976-a7d7-4d68-a991-627cc0348144 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.711796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.712509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.712833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.713150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.713404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988eb807-a697-463a-ab62-3abefb826164 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.727627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.729126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:18.743989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:18.745687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:18.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:18.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:18.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:18.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 258/521] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:18.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.757048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.758002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.758461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.758946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.760032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.760551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.762737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.777101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 264/522] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:18.778359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.803274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.882106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd476201-87d5-4e51-917e-ac3a9ae4e355 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.906758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f78d-dc6f-41e9-ab03-8039881c2ab5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.918485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.933666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.934510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.934986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.935432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.936503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.939024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.958938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.959564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.959847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.960125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.960778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.961081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.962553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.976833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 265/523] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:18.977699 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.042290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 259/524] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:18 2022] POST /identity/v3/users => generated 247 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:19.043470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.111097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217e081-d17b-429e-a64a-7c910f9eb6f2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.164700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.165347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.165640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.165908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.166736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.167047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.168669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.184654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.194701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 266/525] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.194701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.200281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47002-8560-48b8-9a58-397a2f06f27c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.241149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.247302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.247687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.247969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.249236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.249236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.249236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.251664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.265456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 260/526] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.266771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.349881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203b44-cc4e-4d37-aabf-28575d434bcc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.396286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.397745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.398262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.398758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.403084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d3ba1-66b5-45cc-ac97-ad8815b37e17 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.412890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.415455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:19.437474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:19.438601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:19.439711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:19.443439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.444088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.444365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.444635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.446239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 267/527] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:19.447379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.454057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.455618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:19.471165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:19.473263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 261/528] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:19.474554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.500368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.587003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.606561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859efcc-db36-4fe0-b794-7de4f0e9705a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.637716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec6177-db4f-4276-8c69-ac71c3b88083 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.669574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.670274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.670772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.671185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.672527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=9a67c22370884cefb7795cc9eb8b2f8b) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.672934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.674622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.690693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 268/529] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/role_assignments?user.id=9a67c22370884cefb7795cc9eb8b2f8b&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.690693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.709808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.711344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.712084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.712582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.727564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.731054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:19.754223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:19.756140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:19.758643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 262/530] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:19.766628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.895553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507b25-7b6f-4bbc-b861-2a61ac27d223 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.925954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.952777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3af3-8750-41f7-a44f-7b150c9e6bcc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.960595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.961785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.962451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.963086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.964436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.965358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.968140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.991398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 269/531] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.993504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.993896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.994522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.994832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.995104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.995859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.996157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.998242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.020043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.022492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 263/532] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:20.023854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.133746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f7661-4aa3-42c2-9b3a-5e2161191a69 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.172977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.174490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.175525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.176068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.177028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.177844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.179841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.183024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.189117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 264/533] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:20.191319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.193900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a13-573a-4906-acd4-42da9e121978 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.247126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.248375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.248949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.249895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.251018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.251581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.253790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.270415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 270/534] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:20.278343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.435888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf2189-9a04-4a1e-8d9f-cf59ffdbcefb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.476448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.502082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.504508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.507694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.508636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.570747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c1b5d-a6d1-47f8-a377-5a9fcc13888f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.651056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.656019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.659772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.708464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.709403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.709887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.710370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.711812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=d94c7f51f5d5478f858a8eb47cd89257) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.712368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.715280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.726968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 271/535] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/role_assignments?user.id=d94c7f51f5d5478f858a8eb47cd89257&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:20.887876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 265/536] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:20.897617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.148646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75818205-e6a5-4ee0-8004-0c75e96b76c3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.217142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.218234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.218772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.219359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.223528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.248701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.251696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:21.292046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:21.293172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:21.294166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:21.294166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:21.294166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:21.294566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 272/537] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:21.303215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.469245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.473208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2454-30ae-4b8f-8d4d-76f8ac1e5690 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.531677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.532317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.532594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.532867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.533555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:21.534473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.535967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a4f31-8090-4fe5-b5fe-c9cad524c96f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.560447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 266/538] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:21.568884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.642070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9cca-0397-41ee-9190-c175bf33a61a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.642412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9cca-0397-41ee-9190-c175bf33a61a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.642619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d9cca-0397-41ee-9190-c175bf33a61a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.645039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 267/539] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:21.656132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.656437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.656680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.707781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:21.709009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:21.828654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccde4e-a01a-42b6-976b-2c45a66ac3a3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.882682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.882682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.883165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.883658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.899290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.899842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 268/540] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:21.902197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:21.925595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:21.926549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:21.927447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:21.928335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 273/541] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:21.936469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.076233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.098926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1521c6-07e3-4f2a-9a41-dd8fb94442a0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.148758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.149626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.150056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.150503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.151401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:22.151857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.153518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec75412-b7b3-4ef6-a398-2abda4e0d6f2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.183186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 269/542] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.193290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.304089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.369747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427221e-f60a-4e8d-942a-ab6e1cf6b827 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.370358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427221e-f60a-4e8d-942a-ab6e1cf6b827 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.370879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427221e-f60a-4e8d-942a-ab6e1cf6b827 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.373554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 270/543] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:22.387379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.388315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.388809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.423356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:22.423983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fc166-43d6-484b-8dcd-91fef52bdd69 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.425742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:22.465391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.466403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.467088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.467568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.468539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=06a32a4ecd71495785ec867dafc1c8cd) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:22.469051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.470788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.477558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 274/544] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/role_assignments?user.id=06a32a4ecd71495785ec867dafc1c8cd&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.478791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.567595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 271/545] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:22.599974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb4443-4b0a-4e91-b599-35fafd7399f5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.614261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.642479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.645292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.645292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.645292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.671773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:22.678168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:22.679310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:22.680245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:22.682254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 275/546] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:22.691655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.795322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.821722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb1dfa-7fe3-4fc1-a415-c685acbcbb43 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.864215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.865159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.865594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.866020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.866939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:22.867429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.869146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.879402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 272/547] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.887202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.065076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36293d-7d2c-4e32-a1dd-01141e392e59 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.156816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.158001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.158516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.159619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.175796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.189635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.189635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:23.244010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:23.245796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:23.248494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:23.248494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:23.248494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:23.252109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 276/548] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:23.260826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.313227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.431320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a487b4-83e2-48e8-bac1-688178e0cb7f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.457412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e9f3-deec-4b83-b7e2-c9c253904e7a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.496995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.498750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.499572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.500308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.516930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.529831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.531608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.531608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.532100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.532505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:23.534465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:23.535276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.543120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3ad61963-da7f-436b-9a8d-50c08726c53c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:23.548790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:23.551330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:23.552982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:23.564829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 273/549] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:23.567850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc5b1-29ff-42ab-8ed0-e5846c07a73e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.571439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc5b1-29ff-42ab-8ed0-e5846c07a73e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.575392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc5b1-29ff-42ab-8ed0-e5846c07a73e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.576510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 277/550] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:23.578392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.578884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 274/551] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:23 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:23.587619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc5b1-29ff-42ab-8ed0-e5846c07a73e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.716308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc5b1-29ff-42ab-8ed0-e5846c07a73e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.750087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.758637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.759331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.759609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.759881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.769392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.771163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4d320fdf-5199-404d-9635-ea7ebfd35675 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:23.786337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:23.788447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:23.790469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:23.792606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 275/552] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:23.794165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.794452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.794771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.796332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fc80f-e34c-47b2-a817-705e3f31828c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.816619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:23.818746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:23.876982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.878085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.878585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.879095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.880380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:23.880845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.883673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.903055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 278/553] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:23.905022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.947813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.974108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 276/554] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:23.976306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.157166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8113c3-ac17-4a4c-8c0f-95b4f0b37b48 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.230559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eba39-c7b9-49ef-a388-bcdfa09f69a6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.269996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.270632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.270950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.271231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.271993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:24.272299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.273768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.311370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.312945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.313705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.314440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.342686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 279/555] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:24.348837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.352042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.364309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.375173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:24.402709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:24.405749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:24.407557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:24.407557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:24.407557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:24.407557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 277/556] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:24.407557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.521394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.588256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ee2d-4d73-4d10-90b5-d57992436d6b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.655418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e4fb1-62fd-48e6-b81a-4628409865c2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.656742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.657875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.658471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.659101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.713318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.714760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.715189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.715639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.728810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:24.734933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:24.737240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 278/557] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:24.741617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.792150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.795007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf11386-6dfc-40e9-847a-315d1eccf82f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.797418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.913556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 280/558] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:24.914876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e69-6f58-4345-a4d8-6de5e3fb7e25 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.915162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e69-6f58-4345-a4d8-6de5e3fb7e25 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.915459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e69-6f58-4345-a4d8-6de5e3fb7e25 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.916733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 281/559] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:24.920046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb38e69-6f58-4345-a4d8-6de5e3fb7e25 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.926785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2468-2e3e-406c-91f9-45ee8828c02d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.930284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.984234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.984848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.985121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.985387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.986033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:24.986326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.988266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51670c9b-f259-4a01-a376-36e560cd3ca8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.008075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 279/560] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:25.008075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.008075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.008075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.026532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:25.027876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:25.105011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb38e69-6f58-4345-a4d8-6de5e3fb7e25 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.108226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.154686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 280/561] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:25.154686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.182273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.183675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.184345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.184939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.186642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:25.187571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.190721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.208851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 282/562] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:25.210562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.306928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.320409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c80123-9c11-46c0-866d-47aeccf3c83f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.375321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.376216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.376694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.377196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.380956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28515b75-069e-4a20-aa08-52d0257a9986 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.393886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.396503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:25.422007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:25.423993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:25.425828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:25.425828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:25.425828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:25.427065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 281/563] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:25.428407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.447848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.449270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.449812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.450269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.465431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.467992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-5c554750-043a-46fa-a8e1-b66d6b8e2f14 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:25.489188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:25.490229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:25.491215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:25.491786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 283/564] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:25.493563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69f59-98d7-49cf-aa6b-6584bf9df51b None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.494051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69f59-98d7-49cf-aa6b-6584bf9df51b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.494792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69f59-98d7-49cf-aa6b-6584bf9df51b None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.497556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 284/565] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:25.499003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe69f59-98d7-49cf-aa6b-6584bf9df51b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.536548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.593695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84db93-0465-45fa-a524-ff6b1c351a14 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.629043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.629944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.630215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.630490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.641795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.643855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.646721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:25.659507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:25.662231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:25.668058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:25.668058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:25.668058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:25.668395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe69f59-98d7-49cf-aa6b-6584bf9df51b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.668516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 282/566] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:25.668516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.748383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.749093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.749539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.749976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.751084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:25.751575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.754002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c2a67-a9fd-4938-a23d-3142e88de23f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.771687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 285/567] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:25.773183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.773582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.773973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.794594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:25.796161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:25.797882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b871-13c6-42be-9961-2e8f48cec05f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.839591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.840341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.840705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.841066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.841958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:25.842405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.844013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.866231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 283/568] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:25.867673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.873537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 286/569] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:25.875140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.906894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.971136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.035493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdba4-7c9f-4263-ad31-2c3143f81c4e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.037906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9cce-b728-404b-9f1d-65eb33cc826e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.075897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.076536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.076808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.077137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.077742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.078081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.079450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.081006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.081681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.081960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.082227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.094276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:26.101494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 287/570] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.101494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:26.101809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.103530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 284/571] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.197266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.227766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94bdda-feae-468a-967d-6d2fc7547788 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.271955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7c5a0-bd5e-4baa-9f0c-42fe34f18de4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.279303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.280510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.281095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.281541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.289291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.289715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.292075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:26.311870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.313663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 288/572] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.316164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.325561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.326384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.326847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.327207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.340769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.342286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:26.358328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.360483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:26.362836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:26.362836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:26.362836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:26.363182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 285/573] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.364597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.444118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb76666-3dcf-448f-899b-eb4f76a416fa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.497578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.498479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.498969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.499449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.503255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.510752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f3ba-8793-45b4-95e3-5ddb24bc8227 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.513498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.515622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:26.531679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.532705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:26.534184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 289/574] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.534814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.560679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.561287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.561551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.561824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.562494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.562815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.564198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.573837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 286/575] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.574909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.617137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.648897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ef5d7-63b3-491d-a319-56739841b5db None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.697186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.698277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.698687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.699200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.710964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398bc34-0f37-466d-a180-aac69cf741be None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.714106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:26.719813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.723816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 290/576] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.724692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.762481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.763057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.763336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.763612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.764324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.764626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.766112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.782147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 287/577] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.783078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.795831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.848806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88fe0-7082-42a5-93c6-f3898029b719 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.894731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.897650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.898493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.898904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.899373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.900171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.900645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.912036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.912187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11c66-48fe-47d6-9475-9d0202ea8ea9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.915929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 291/578] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.916781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.964153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.965022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.965609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.965609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.966300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.966602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.968100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.991910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 288/579] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.992792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.046722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c8451-26e2-4672-ad41-9dab495d5ceb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.065980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.091494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.092189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.092476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.092755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.105259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.106902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:27.124218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.125319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:27.126253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:27.126253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.126253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:27.127053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 292/580] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:27.128194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.132164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd6550-deff-4631-85e9-89da118ff0d3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.192295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.193251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.193689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.194126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.195271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.195760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.198296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.218091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 289/581] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.218091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.236698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.261745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83dae9-60c3-458a-8480-d3cae5e9cdc9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.307948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.308959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.309443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.309932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.311456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=a3b614a59a2e40c09dc87e99a717caf3) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.312030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.315390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.326125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 293/582] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/role_assignments?user.id=a3b614a59a2e40c09dc87e99a717caf3&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.327576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.371102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.384347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af796-d762-4846-a8c5-7ccfbfa5915a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.447173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.448370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.448962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.449517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.463924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.467108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:27.490343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.491344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:27.492246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:27.492682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 290/583] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:27.493740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.500123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc93a-ccf2-4521-9b33-79080b13d08f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.575329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.576574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.577028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.578066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.599380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.602274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.608775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:27.626535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.628116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:27.629025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:27.629025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.629025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:27.634834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 294/584] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:27.636162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.664069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180449da-fc65-41df-b950-5b0942b24c8d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.723502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.724518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.724968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.725401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.726516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.727042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.729370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.749166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 291/585] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.750604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.777323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.802246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69893e-b612-46e0-bf1f-32362e2bc22c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.860020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.860881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.861870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.862319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.880944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.883446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:27.907166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.909121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:27.910568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:27.910568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.910568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:27.910568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 295/586] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:27.910568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.936267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06861fa-545b-4569-848b-3ec062901dc9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.971836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.984826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.986716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.986892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.987387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.988721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.997285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.997285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.022436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 292/587] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.023980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.060452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937bd4-bd72-4d88-abc4-b9798eb3d71a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.104665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.105633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.106031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.106364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.107141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.107509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.108958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.113802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.133205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 296/588] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.133205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.216973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4d993-1a0c-430a-9842-82e72f66599e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.267022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.267713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.267981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.268259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] PATH_INFO: `/v3/users/glance` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.277824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.279373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:28.301029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:28.311075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 293/589] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:28.312150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.314694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a4b39-e789-4e74-9830-7f092faace2b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.371041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.381448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.382596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.383097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.386175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.386175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.386175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.388200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.415925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 297/590] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.415925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.453700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2909f-e735-4d7d-ba8b-aa397836331d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.514244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.515172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.515583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.516041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/9a67c22370884cefb7795cc9eb8b2f8b/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.520521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.546580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=9a67c22370884cefb7795cc9eb8b2f8b, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.548456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.612399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bec59-0c4d-4068-8fd8-c9660999cd19 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.624622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 294/591] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:28 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/9a67c22370884cefb7795cc9eb8b2f8b/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:28.625703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.695312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.696426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.696925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.697399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.711551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.714853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:28.739653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:28.741909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:28.743688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:28.749226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 298/592] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:28.750864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.851981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.858331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9a54-92c4-49d2-894c-3c0b699ee18e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.927844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.929003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.929588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.930182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.931873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.932675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.935629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.961837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e294-177a-405d-94c6-c186d9573f9b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.973294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 295/593] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.975027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.991612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.033439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.035012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.035802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.036523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.052322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.055586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:29.082031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:29.085372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:29.087364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:29.091460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 299/594] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:29.093289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.177779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89944-f4aa-42dc-a8c0-a55dfbf0a825 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.225945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.226759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.227042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.227319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.228482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:29.228807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.230272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.248379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 296/595] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:29.249338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.266346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2240d1-731b-4c41-bc13-28dbd67eb438 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.315300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.332087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.333011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.333444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.334534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.350512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.353065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:29.374547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:29.376503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:29.379886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:29.379886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:29.379886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:29.379886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 300/596] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:29.379886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.416633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.449111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-9ed9-43c2-b198-007ca01f5972 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.515193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.516153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.516637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.517124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.518554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:29.521914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.521914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdfd83-0e7e-4cf5-bff3-9352e4079fce None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.556248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b2b22-9d94-45e1-8491-db967a7ec814 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.558707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 297/597] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:29.560943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021583-6044-41c0-94de-b8f3bf71757a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.561430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021583-6044-41c0-94de-b8f3bf71757a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.561880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021583-6044-41c0-94de-b8f3bf71757a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.566825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 298/598] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:29.566825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87021583-6044-41c0-94de-b8f3bf71757a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.616979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.617720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.618000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.618276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.641456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=0880178d7ea842afba2a0a4b8bd6d842, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.643433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.689800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.717757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87021583-6044-41c0-94de-b8f3bf71757a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.755657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.756361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.756701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.757036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.757758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:29.758118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.759656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391b8c6-287f-419f-9ab0-a11253d5faef None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.777370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 301/599] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:29 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:29.777370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.793128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 299/600] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:29.796590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.797445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.798045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.827786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:29.829375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:29.949581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.962281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 300/601] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:29.964435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.996534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172deb-6ccf-4db4-b6ad-3df834c9f147 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.078549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.080180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.080759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.081373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.104921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.109486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:30.142718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:30.144633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:30.146632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:30.146632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:30.146632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:30.149000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 302/602] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.149191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.159575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.208833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0e64c-f80b-4b75-b5f3-d8e87547c989 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.338605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.340390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.341303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.341989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.357236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.360703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:30.384326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:30.386348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:30.388360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:30.392845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 301/603] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.404195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.433840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecea45-78f1-4080-bf15-42d033bfff47 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.465530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.485059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.486168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.487583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.488021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.489790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:30.491450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.493104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.523332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 303/604] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:30.531378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.665597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089b05b-f9c0-45a1-b30f-9f106c96c82c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.678616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7470d5-4fcb-4fcc-b919-00ec2bacc98e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.724459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.727951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.729733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.730308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.730607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.742846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.744708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.750397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.752472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.753306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.754585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.756051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:30.756875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.759422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:30.761992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:30.763937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:30.765623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:30.769611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 304/605] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.771599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.784260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 302/606] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:30.785474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.874708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.945829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69160b8-db0d-486f-8376-a699faa84451 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.989732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63020f-00fa-4d3b-a219-c033489470a5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.993786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.994585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.994808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.995098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.995822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:30.996129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.998722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.023131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 303/607] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:31.023131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.074133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.075001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.075201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.075522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.088669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:31.094076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:31.095669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:31.096731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:31.097757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:31.100827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 305/608] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:31.101916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.191984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a0795-32ba-4e7e-9e46-c15ee18d1da3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.237369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.237979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.238259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.238537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/d94c7f51f5d5478f858a8eb47cd89257/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.257218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=d94c7f51f5d5478f858a8eb47cd89257, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.259522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.269377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5956-8cc3-429e-8033-5635a2d9a359 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.286497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:31.313833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.314533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.314850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.315123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/06a32a4ecd71495785ec867dafc1c8cd/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.333507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=06a32a4ecd71495785ec867dafc1c8cd, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.336466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f76f8c-5e3b-455c-a16a-8a28d84a101e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.359070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 304/609] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:31 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/d94c7f51f5d5478f858a8eb47cd89257/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:31.360174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.451242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 306/610] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:31 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/06a32a4ecd71495785ec867dafc1c8cd/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:31.453833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96f0e-1d6f-41be-9eda-0c26214d5bf9 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.454445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96f0e-1d6f-41be-9eda-0c26214d5bf9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.455567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f96f0e-1d6f-41be-9eda-0c26214d5bf9 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.465299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 307/611] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:31.468058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.468658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.469334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.531770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:31.534520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:31.559182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:31.564587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc2b-2efd-407c-ade7-ed05ddfcf71d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.640433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.641872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.643115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.643735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.646064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:31.646926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.649552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.673639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 305/612] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:31.689096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 308/613] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 08:35:31.692682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.854967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b8e17-b997-4870-a887-2b9130b3f03f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.893015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.893660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.893934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.894202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.906070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.907610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:31.923249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:31.924295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:31.925199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:31.925199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:31.925199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:31.925199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 306/614] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:31.936485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.055691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.069568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765961f-aad8-463c-a5d0-88c416ea7036 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.108995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.110378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.111528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.112005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.112861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:32.113365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.114955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.128845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 309/615] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:32.136886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.151082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.176225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.302414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ddaf2-6fb3-43f7-83e6-94701562c99b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.360202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9ddf-3ac6-4034-9c5d-ddb209bf5355 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.408359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.409019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.409306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.409584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.423935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.424945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.425405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.425848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.440319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.443122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.443817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:32.449091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:32.450264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:32.451331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:32.455617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 310/616] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:32.459686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:32.467009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:32.468919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:32.472284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 307/617] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:32.480686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.485440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.595616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.626561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb044c1-ba08-4f86-b414-6929d7464d97 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.633010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d87de9-e5f6-4f9c-82ce-2c18b4eed3e9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.663810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.664586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.664969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.665320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.666070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:32.666441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.667942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44519-bf75-4396-8d31-a597801b001d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.671658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.672369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.672724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.673058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.674466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:32.674857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.676385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa521f-198f-40b7-9673-f565411ec278 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.681991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 311/618] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:32.683469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fed4-34eb-45b9-937a-00c294dcd159 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.683762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fed4-34eb-45b9-937a-00c294dcd159 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.684042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fed4-34eb-45b9-937a-00c294dcd159 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.689776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 312/619] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:32.690712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042fed4-34eb-45b9-937a-00c294dcd159 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.696802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 308/620] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:32.698113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.698402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.705068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.737249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:32.739772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:32.766966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.801050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042fed4-34eb-45b9-937a-00c294dcd159 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.839244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.840060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.840450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.840816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.841297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 309/621] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:32.842284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.876397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.878160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.912116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:32.912756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:32.914513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:32.916274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:32.916274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:32.916274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:32.923256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 313/622] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:32.929560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.963527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ba5fe-6d0e-4343-a1e5-28540cf71e62 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.007084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.008426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.008709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.009128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.022350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.023919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:33.042112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:33.045558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:33.049086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:33.049086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:33.049086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:33.049086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 310/623] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:33.049913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.182844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189441-00c2-426c-8d53-71ab4b326298 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.188795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576bcd-7a65-43db-b101-0aa6c8464fbe None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.215243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.231532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.232155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.232440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.232704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.233341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.233659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.235058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.251489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 311/624] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.256041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.266464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.283973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.284557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.284848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.285100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.286465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.286828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.289344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.305034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 314/625] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:33.306001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.429636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338db4-ea03-436c-adf3-a338724a8936 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.441977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40c37-474f-4854-95c1-9efb7de1bb29 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.474423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.475137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.475417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.475689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.485049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.486563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.508953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.509593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.509867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.510138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:33.514575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:33.516571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:33.519562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:33.526679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 315/626] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:33.526679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.531221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:33.537768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:33.538810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:33.539670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:33.551200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 312/627] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:33.552383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.591206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.592593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.659214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d37612-0680-4d12-b92f-2ceb27898b36 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.672342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80a9f-e4e8-44a3-9834-0d418c82e2bc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.700676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.701648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.702272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.702898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.711333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.711548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.711954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.712233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.712541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.713357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.713662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.714739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.715597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:33.728528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:33.731393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:33.732368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 316/629] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:33.734220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.735007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 313/629] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.737298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.838532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.852718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c91d-1d4d-4e34-8f58-9cb313d3c905 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.862960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9788e-2276-41a8-acfa-3a566280dbb6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.864962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.890863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.891622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.892301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.892552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.893299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.893604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.895109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.901051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.901962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.902250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.902527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.903313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.903827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.905917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c9cf-0971-48f6-9970-b9cbc2e15f5c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.921244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 314/630] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.921244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.922878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 317/631] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.924501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505f517-0046-4bf9-865e-f33e1423f964 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.924834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505f517-0046-4bf9-865e-f33e1423f964 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.925237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505f517-0046-4bf9-865e-f33e1423f964 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.927828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 318/632] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:33.929187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505f517-0046-4bf9-865e-f33e1423f964 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.078336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8227cf-f7bf-4a3c-9fc1-61cc3d889ed8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.088549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505f517-0046-4bf9-865e-f33e1423f964 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.120978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.121917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.122216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.122478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/a3b614a59a2e40c09dc87e99a717caf3/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.147368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.147983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.148263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.148546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.149951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=a3b614a59a2e40c09dc87e99a717caf3, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.152262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.155735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.184732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.225856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.227372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b89f6e-57f7-4213-af8b-e8921a70c334 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.235029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 315/633] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:33 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/a3b614a59a2e40c09dc87e99a717caf3/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:34.236854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.404191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 319/634] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:34.406812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.407593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.407911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8605f6-62f0-4359-9000-198d761629d6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.408403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.437991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:34.440489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:34.448797 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.449904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.450409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.450940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.470149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.472524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b7052a86-6a19-4e38-852b-d3723fcaba42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:34.498400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:34.499501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:34.500395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:34.500395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.500395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:34.500395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 316/635] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:34.503235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c516f3-ffef-4aa9-a8f7-47b23198e8d7 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.503743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c516f3-ffef-4aa9-a8f7-47b23198e8d7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.504243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c516f3-ffef-4aa9-a8f7-47b23198e8d7 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.507151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 317/636] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:34.508456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c516f3-ffef-4aa9-a8f7-47b23198e8d7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.531155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.544622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 320/637] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:34.547655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.598414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.681453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23566-e0d6-4feb-a6b8-922723dc3b06 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.693943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c516f3-ffef-4aa9-a8f7-47b23198e8d7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.726999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.728010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.728503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.729172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.730089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:34.730979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.732590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9241cc-21c7-4b38-a783-472da04a4034 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.750584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 321/638] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:34.753207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.754189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.754705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.768800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.769888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.770387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.771618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.777862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:34.779541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:34.789952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.792169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:34.813385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:34.814469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:34.815520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:34.815520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.815520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:34.815520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 318/639] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:34.816275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.846689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.870567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 322/640] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:34.871350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.970311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.976641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0190-a425-490a-b952-8ade09934fc9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.044260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.044743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.045160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.045450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.060183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.062855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:35.085712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:35.087498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 319/641] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:35.090321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.107292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f844087-b498-4bff-926f-d5ed1001373d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.185043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.186898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.187514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.188008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.189187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.189728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.192205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.219157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 323/642] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.219157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.238052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81404b8-f623-4183-b3a4-0204ca02ffc3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.289446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.292827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.293488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.293763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.294120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.307250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:35.312656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:35.315076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:35.316876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:35.319696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 320/643] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:35.319696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.354016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.385780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342883f9-5c37-493c-a0da-e3c7638e9250 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.429515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.430129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.430408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.430697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.431445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.431745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.433197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.452324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 324/644] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.453195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.461829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6425-f06e-431f-b43d-44eaacd50720 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.504664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.505771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.506241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.506578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.521529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:35.526402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:35.527545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:35.528729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:35.533149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 321/645] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:35.533988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.585624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566ab9-74ba-4374-82f9-ed6e47483c0c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.587424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.630719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.631503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.631784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.632052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.640039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.641588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.646536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:35.655321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:35.657270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 325/646] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:35.658182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.671600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844bc4b-87d3-48d8-93f3-c49b5b4a7504 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.718497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.719223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.719513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.719799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.720489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.720787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.722220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.732819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 322/647] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.733809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.788837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ce3-1727-4609-a733-510957785cb6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.832522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.833565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.833964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.834306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.863031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933beb1-052e-43d7-9ba5-91ded6f6fc15 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.874625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.910225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.910954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.911244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.911507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.912205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.912504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.914060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.924987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 323/648] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.925688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.930175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.944429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.946064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.058463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeedb8f-2efe-42b4-822a-b9e5981168da None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.088245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 326/649] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:36.093364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.105684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.106758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.107217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.107702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.109069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:36.109697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.112997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.134443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 324/650] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:36.135657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.202836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:36.232738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:36.256653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a687b8-e5a3-4b27-b7d9-329c32740b4d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.280833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186ce0d-fda5-4715-b8df-c9701a89f60c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.310428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.311472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.312291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.312706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.331178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.333662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.333871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.335227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.335715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.336208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.350847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.352668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:36.353245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:36.355092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:36.356968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:36.356968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.356968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:36.356968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 327/651] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:36.356968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:36.385009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:36.386924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:36.389413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:36.389413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.389413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:36.389413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 325/652] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:36.389413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.528510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a9ae8-5ac2-4b70-abd8-a283e0c7afb6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.549759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:36.551996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6052-ba52-4b7d-a090-13618dd16cb3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.577316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:36.579875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.580523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.580804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.581078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.593589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.595508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.596882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.597489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.597763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.598358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.598786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=9a67c22370884cefb7795cc9eb8b2f8b) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:36.599087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.600685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.607502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 326/653] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/role_assignments?user.id=9a67c22370884cefb7795cc9eb8b2f8b&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:36.608472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:36.610611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:36.612024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:36.612964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:36.612964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:36.612964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:36.612964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 328/654] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:36.612964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.757484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920803dc-cb99-43c1-9be0-889711ea54b0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.807123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca5d9-c0c3-4e9a-b09b-92a335bf99c3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:36.821714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.822741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.822741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.822976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.825343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:36.825343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.826225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1987363-d699-4237-9bbb-352057c0f785 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.848411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 327/655] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:36.848411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395729a-3e42-42c0-a748-a9ba46da8894 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.848411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395729a-3e42-42c0-a748-a9ba46da8894 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.848411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395729a-3e42-42c0-a748-a9ba46da8894 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.850789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 328/656] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:36.852084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395729a-3e42-42c0-a748-a9ba46da8894 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.862682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:36.863367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:36.863719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:36.863997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:36.864660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:36.865006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:36.866468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:36.884422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 329/657] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:36.884422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:36.916205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:36.919031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.014177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe679-953c-4846-8f68-127ed52b84ec None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.053532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.054235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395729a-3e42-42c0-a748-a9ba46da8894 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.054350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.054636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.054942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.063176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.064681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-006d2b87-ebee-4901-8e0a-ecbfe97c4364 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:37.078742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:37.079782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:35:37.080737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:37.084897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 330/658] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:37.086386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.086692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.086970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.107044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:37.108174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:37.120185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.120969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.121683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.121951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.123211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:37.123754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.126508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.150630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 329/659] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:37.152295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.200337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 331/660] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:37.201677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.246607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.354613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.359626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316404a5-ccfb-460a-92a9-833d59871285 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.366609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618432b-95da-4464-9763-8e9d0cdc4dbf None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.405817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.406709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.407173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.407597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.408627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:37.409103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.410847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.423492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.424766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.425402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.426017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.432785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 332/661] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:37.433997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.445188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.445188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:37.470386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:37.471462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:35:37.472413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:37.475900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 330/662] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:37.475900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.525102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.554148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7263f6f-3227-4dbc-b384-3bde54a47fc5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.599354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.600390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.600901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.601386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.611238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.613192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:37.627409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:37.628621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:37.629745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:37.631526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 333/663] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:37.632489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.648543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024c0d5-3f68-4f81-851e-db998e2d96fb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.694432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.695357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.695790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.696226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.728116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.743249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e37e7-83fc-4fb0-adf4-31408c1e0eed None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.773379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.774889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.785851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.787142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.787862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.788349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.789510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:37.790387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:37.792510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:37.794831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.818902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 334/664] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:37.819785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.883990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 331/665] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:37.885249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.943285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783e04-515c-4539-a3dc-2c141b73cd65 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:37.960117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:37.984564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:37.985244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.985532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.985804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.999740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:38.005243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.006241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:38.011913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:38.012568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138496-980a-4682-a3bf-6b336ee707f1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.012646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 335/666] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.012646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.051651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.052274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.052549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.052823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.053542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.053837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.055460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.070782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.075203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 332/667] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.076155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.126198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea66b-352d-4329-940e-4c5f3fb04785 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.173860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.174990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.175503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.176008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.206511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acb7-3762-4788-a6a7-38b4e5b6591b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.229816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.247220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.247858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.248175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.248414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.261343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.262849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:38.279028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.280063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:38.281074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:38.281074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.281074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:38.281074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 333/668] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.281315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.291304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.293085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.378956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.412616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e331d0-b596-4e36-b4c4-307176255436 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.452715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.453704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.454207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.454738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.455970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.456504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.464076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 336/669] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:38.464076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.464465 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.482909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 334/670] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.482909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.531618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.599407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586cf3-737c-49d9-a006-96d77709a499 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.630460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d32826-2a59-4cb0-b4d3-3fd3480de845 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.643457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.644247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.644362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.644605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.645347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.645653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.647135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.653795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 337/671] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.654339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.676604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.703270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.704335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.704822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.705304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.706213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.706794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.708497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.735310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 335/672] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.736480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.803174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c96d-6cb6-4aa2-aa36-721b92d17775 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.857225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.857883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.858168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.858443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.872521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.874006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.880081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.888638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd75455-cb5c-4ab2-8ba5-222bf958c72d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:38.889984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.895152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:38.896312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:38.896312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.896312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:38.896312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 338/673] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.896312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.947243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.948155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.948609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.949056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.967493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.970697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.974442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b9d6921d-786f-45d4-a942-e514102e5cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:38.993522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.995410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 336/674] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d1ef-be96-46dd-a24b-699af7eaf037 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d1ef-be96-46dd-a24b-699af7eaf037 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.997190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d1ef-be96-46dd-a24b-699af7eaf037 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.999011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 337/675] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:38.999860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6d1ef-be96-46dd-a24b-699af7eaf037 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.034983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0dd4d-ac63-4d86-a050-075a0adef84d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.090409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.091299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.091716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.092140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.101102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.102766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:39.115334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:39.117079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 339/676] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:39.119696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.173770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6d1ef-be96-46dd-a24b-699af7eaf037 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.224303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.225262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.225728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.226111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.226991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.227549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.230170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb746f2b-4ee3-4540-b020-3965b1c159a7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.258694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 338/677] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.258694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.258694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.258694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.276328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad84b0-fef0-4207-9913-137fc0064712 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.277883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.289664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:39.291828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:39.321759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.330313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.331077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.331362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.331668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.332324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.332702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.334546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.353584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 340/678] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.354689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.401891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 339/679] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:39.402980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.497089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780b88d-752c-4790-8015-af1ba7b2312f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.522432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3a5f2-42eb-44ad-a4c0-ea093735ab1c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.543710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.544720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.545214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.545719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.546737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.547352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.549077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.566352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 341/680] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.567781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.582035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.582410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.582734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.583017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.586276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.593347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.593840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:39.606960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:39.608618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:39.610000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:39.612966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 340/681] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:39.613945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.640198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.701633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6036d2-20a1-4831-9c8d-f9a7000cc166 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.732457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69aab1-ac0f-4846-acba-c1d6987f5a29 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.742340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.743024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.743308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.743582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.753107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.754498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:39.767223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:39.768350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:39.770248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 342/682] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:39.771508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.773320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.774043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.774322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.774592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.775349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=06a32a4ecd71495785ec867dafc1c8cd) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.775659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.777191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.790703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 341/683] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/role_assignments?user.id=06a32a4ecd71495785ec867dafc1c8cd&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.790703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.884363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.931838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feff75-a4b6-4146-9a22-d1aedf672255 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.938993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e072-766e-45ae-b1ba-1ea7f010a59c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.947104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.975625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.976493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.976886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.977268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.978280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.979128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.980778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.988842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182f619-5829-4456-9292-f752be5084aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.989770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182f619-5829-4456-9292-f752be5084aa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.990195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182f619-5829-4456-9292-f752be5084aa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.990638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182f619-5829-4456-9292-f752be5084aa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.992090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182f619-5829-4456-9292-f752be5084aa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.994298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182f619-5829-4456-9292-f752be5084aa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.003276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 343/684] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:40.004611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e182f619-5829-4456-9292-f752be5084aa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:40.012850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:40.013921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:40.014849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:40.017112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 342/685] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:40.018603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182f619-5829-4456-9292-f752be5084aa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.146284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e303-f576-441d-9f59-ea8de8963946 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.190679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182f619-5829-4456-9292-f752be5084aa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.221663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:40.234120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.235723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.235837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.236376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.236823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.246839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.246985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.246985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.247151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=d94c7f51f5d5478f858a8eb47cd89257) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:40.247151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:40.247295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:40.247368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.247437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:40.247437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.254781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 343/686] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/role_assignments?user.id=d94c7f51f5d5478f858a8eb47cd89257&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:40.255841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.286383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 344/687] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:40.300878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.363198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:40.477877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7feaa3-b752-462d-a36b-3c5dba7602ce None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.488764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c999-09cc-4a57-939b-a48a8dcfebd9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.526700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.527597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.527739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.528009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.561800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.564517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.565279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.566049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.567535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:40.568478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:40.571371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.581336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 344/688] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:40.588896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.622639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:40.624122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.662724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:40.716570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:40.787573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 345/689] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:35:40.794269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:40.801310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dd89-487f-4682-a97a-80f317bb9e58 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.870910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.872415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.873139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.873835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.875208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:40.876054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:40.878795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6a73c-d0e8-4494-8648-9c81c33be336 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:40.889353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 345/690] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:40.892234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392957d-1aca-40c4-a0b9-f98242a07355 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.893110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392957d-1aca-40c4-a0b9-f98242a07355 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.893898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392957d-1aca-40c4-a0b9-f98242a07355 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.897308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 346/691] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:40.904876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.905383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.905880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:40.915972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c67dda-44c3-4b41-95f9-8b5c517d6c98 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:40.938144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:40.939923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:40.966101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:40.981568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:40.985690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:40.985690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:40.985690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.020277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:41.025676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:41.080494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:41.082418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:41.084419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:41.084419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:41.084419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:41.090825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 346/692] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:41.097026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.112691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 347/693] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:41.296700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb51f4-c2b3-4e4b-8aad-e692c1b60921 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:41.361583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:41.362960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.363733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.364392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.365820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:41.366592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:41.369516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:41.394216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 347/694] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:41.399987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.459732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.504615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:41.555185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:41.636630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383c110-5a21-4d67-909a-3ec6bf20461a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:41.694071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2ca0-7de8-452c-9b3c-f0169a228fb0 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:41.705021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:41.705947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.706412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.706863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.721508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:41.723456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:41.746610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:41.747838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:41.749000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:41.756879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 348/695] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:41.762297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.778090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:41.780148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.781736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.782158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.806631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:41.813734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:41.814888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:41.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:41.819911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 348/696] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:41.825581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.978510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:41.982761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da08b8-4ae7-4b18-9edf-e53c61f0882c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.027435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.028079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.028362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.028639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.029354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.029661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.031581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.043069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 349/697] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.049715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603edda-ebaa-4641-8847-b969e26ad44b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.058901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.060591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.139417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.140050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.140394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.140671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.141421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.141722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.143241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.170263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 349/698] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.176763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.199125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659dd1c-cfa1-4c61-ac0b-29ba6ee6d804 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.240259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.240831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.241114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.241390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.256204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.258233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:42.274874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:42.275929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:42.276842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:42.276842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:42.276842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:42.276842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 350/699] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:42.282969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.301135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.404520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697eb3d-ace2-420e-908c-c800fdad593f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.444660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.445291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.445575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.445850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.446826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.447134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.449048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.464173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 351/700] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.470926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.481692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc14711-d090-4974-8211-79c69e7a5c43 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.568873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.569878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.570379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.570890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.571844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=a3b614a59a2e40c09dc87e99a717caf3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.572366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.574045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe14358-581a-48d7-a3f3-c617eb3a3923 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.592193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 350/701] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/role_assignments?user.id=a3b614a59a2e40c09dc87e99a717caf3&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.607109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.631946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84868-5339-4f95-b4f3-c4fd6fd25cd0 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.645710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bde49-f3e9-4c64-95b0-30ef154a52ae None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.646325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bde49-f3e9-4c64-95b0-30ef154a52ae None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.646441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bde49-f3e9-4c64-95b0-30ef154a52ae None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.653741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 351/702] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:42.658187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.658187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.658187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.675014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.676475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.676979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.677467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.686802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.688934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:42.703741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:42.704775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:42.706245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:42.708881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 352/703] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:42.714765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:42.714990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.717657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:42.771058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.839599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 352/704] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:42.900879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6946b-323a-4cdd-9e85-95672bf04553 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.939536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.941993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.942683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.943841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.944132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.944977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.945287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.946786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60eda9-98ae-4475-b0fd-136704bd9d3d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.967082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 353/705] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.975396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.078412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984496e-6d0c-4cb1-b5f7-76e189214c91 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.078795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984496e-6d0c-4cb1-b5f7-76e189214c91 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.078982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984496e-6d0c-4cb1-b5f7-76e189214c91 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.080549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 354/706] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:43.086297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.086565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.086928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.112584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:43.114500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:43.166604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd35fa9-5e8a-4626-9600-018d0cfa8b72 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.207611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.208746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.209381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.209761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 355/707] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:43.210248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.210718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.266321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.343428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.345511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.352161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8323f0-4e1e-4f7d-994b-e3bde76d9502 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.376022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.392537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.402083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.402083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.402083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] PATH_INFO: `/v3/services/network` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.402083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.402083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:43.407662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:43.408764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:43.409673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:43.413416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 356/708] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:43.415421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.532941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22b79-8093-402d-b1b3-f366b36a1750 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.579916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.580614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.580895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.581170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.581861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] RBAC: Adding query filter params (name=network) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:43.582206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.583697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.596880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 357/709] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:43.598348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.617074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 353/710] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:35:43.618539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.678758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.744742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.745717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf3dc3-29ab-4074-b5e5-182a81d0502f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.792782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2110d-d061-484d-b8d8-f6b6d07ef4cf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.806113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.807196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.807677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.808175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.828821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.831537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.834175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.834825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.835105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.835411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.836097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:43.836397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.837790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.850809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 354/711] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:43.850809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3853a302-2ac2-468e-b98e-89271cd572af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:43.851205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:43.852924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 358/712] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7ebf-3a5a-4e63-90de-01de4ba61b9f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7ebf-3a5a-4e63-90de-01de4ba61b9f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7ebf-3a5a-4e63-90de-01de4ba61b9f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.854359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 359/713] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:43.859124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab7ebf-3a5a-4e63-90de-01de4ba61b9f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.989808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.994831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab7ebf-3a5a-4e63-90de-01de4ba61b9f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.016623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc431260-c94f-43ce-a696-2581106346b5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.043852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.044823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.045039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.045295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.046440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:44.047063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.049309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915de0e-7069-486b-9f5e-66c7ad644932 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.058622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.067633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 360/714] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:44.070841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.070841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.070841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.079212 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.080022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.081470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.081874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.097008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:44.098568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:44.098926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.101891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:44.129067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:44.130408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:44.131358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:44.131358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:44.131358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:44.131544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 355/715] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:44.132994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.214971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 361/716] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:44.220122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.308654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4cfe-dd20-4c93-92c0-a68beeb8bbe8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.394479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.404909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e4c6-4e6d-43dd-a1c4-a3e3911be4a5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.416129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.417127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.417608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.420679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.422470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.428739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.455494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.456207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.456611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.456915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.457620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:44.457943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.462854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.477582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 362/717] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:44.478953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.546468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 356/718] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:44 2022] POST /identity/v3/users => generated 254 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:44.618283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a4b24-195a-468f-bddf-1a2486083135 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.696248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.698415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.699237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.699999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.718207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.725729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:44.748272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:44.749895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:44.751566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 363/719] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:44.756064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.804887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.811766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.833518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.922612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503ee71-e5e4-4775-9bc5-59eb69d5e189 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.953471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5468b-ef87-45a5-822f-a4cabbc4c678 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.993122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.994248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.994817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.995213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.995987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.996305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:44.996611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.996854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.996963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.997167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.997833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.999277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ae1d-8f0a-4911-a6ed-648b32186df3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.999411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:45.025637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 357/720] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:45.035740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:45.105470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 364/721] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:44 2022] POST /identity/v3/services => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:45.201130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:45.203785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a6ad8-f176-468f-a383-91b7035b3065 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.204508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a6ad8-f176-468f-a383-91b7035b3065 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.205065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a6ad8-f176-468f-a383-91b7035b3065 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.207140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 365/722] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:45.212300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232718b-fd64-49d5-a3c0-7b2ba5755457 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:45.228049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.228398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.228697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.263903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:45.264899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.265027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.265234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.266549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:45.266901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:45.268496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:45.277498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:45.278876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:45.281597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 358/723] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:45 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:45.408371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:45.415911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 366/724] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:35:45.716942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:45.838223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a3602-817c-4cfe-9c91-734f60115b47 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:45.876288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:45.877305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.877792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.878291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.889759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:45.895581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:45.896613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:45.897521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:45.900287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 359/725] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:45.906014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.087224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423ab5-ce43-4a82-bffd-193b9a19266f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.151930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.153449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.154215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.154895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.156184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:46.156904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.159494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.173155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 367/726] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:46.180096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.203604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.308348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664c668-a026-4bbd-8e9c-f831b3ddd496 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.347318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.348324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.348876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.349359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.361092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.362950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9d36cb07-9fd1-46c4-9867-104a97f3e603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:46.379645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:46.380674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:46.381591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:46.381591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:46.381591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:46.383794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 360/727] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:46.393624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.465575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.508277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd10c-5103-4c9d-8395-f2d7bc4f8d35 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.542332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5322ecc-5a22-4a7f-949a-279742769f96 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.543925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5322ecc-5a22-4a7f-949a-279742769f96 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.544418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5322ecc-5a22-4a7f-949a-279742769f96 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.546410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 361/728] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:46.552582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.553179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.553696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.570455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.571111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.571395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.571672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.572332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:46.572634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.574026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.577818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:46.579517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:46.596800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 368/729] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:46.604678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.655082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.665804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 362/730] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:35:46.773850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d9772-3c49-4219-81e8-5a8522c1c9db None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.803817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.817502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.819444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.819444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.819444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.829937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.832245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:46.847469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:46.848548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:46.849510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:46.854039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 369/731] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:46.860475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.985379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:47.049749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c2cd1-ee45-47c5-8ccd-149842c908b0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:47.120722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:47.122187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.122935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.123971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.125173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:47.125821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:47.127778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:47.150426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 363/732] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:47.159942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:47.227348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b616c2-bc21-4b08-8c92-c3b9ed228d98 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:47.291651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:47.292999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.293666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.294310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.295598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:47.299121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7961-f04b-4ae3-9dbf-6fc123cc6daa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:47.329053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:47.402257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 370/733] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:46 2022] POST /identity/v3/services => generated 240 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:47.435242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aef301-9bed-4858-ab20-7912c4227d2f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:47.454605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45a3b-56c1-4c2a-98f7-f74848402c30 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.454605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45a3b-56c1-4c2a-98f7-f74848402c30 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.454605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45a3b-56c1-4c2a-98f7-f74848402c30 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.457743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 371/734] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:47.471121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.471696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.472923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.500376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:47.502108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:47.552416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:47.553072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.553355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.553628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.576631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:47.618706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 372/735] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:47.659916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:47.661547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393762be-f6f2-4bb3-b842-ce21c0060722 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:47.715399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:47.773615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 364/736] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:47.780032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:47.904594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dbec8-c923-426b-9811-ce16ad6df044 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:47.910986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce26ee-a340-4458-b239-fff4b46d5c6c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.911520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce26ee-a340-4458-b239-fff4b46d5c6c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.911601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce26ee-a340-4458-b239-fff4b46d5c6c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.913237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 365/737] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:47.918577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.918908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.919190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.939795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:47.940753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:47.955928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:47.956979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.957468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.957995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.970787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:47.972787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:47.990794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:47.992271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:47.993186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:47.993186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:47.993186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:47.995598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 373/738] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:47.996990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.016593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 366/739] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:48.017613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.028220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.117779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3af92-1b2a-41c1-9f13-de532438b190 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.179347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.184920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.186015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.186488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.186930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.217057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:48.219208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dbc66-d423-4da5-b75b-52adb574c9ea None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-8974e131-9542-471a-99a7-b2c5bc3edfc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:48.230726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.232305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:48.234024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:48.240681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 374/740] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:48.240681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e94b9-17e3-45df-873b-62c22a131229 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.240681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e94b9-17e3-45df-873b-62c22a131229 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.240681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e94b9-17e3-45df-873b-62c22a131229 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.240681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 375/741] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:48.249358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e94b9-17e3-45df-873b-62c22a131229 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.281844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.282791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.283265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.283978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.284789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:48.285251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.287024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b424c8-b9d9-4036-b8f3-f5d4b24313b7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.305114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 367/742] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:48.307073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.307537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.307973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.332250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:48.333767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:48.377655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.396569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e94b9-17e3-45df-873b-62c22a131229 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.418727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 368/743] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:48.418727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.451516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.452165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.452576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.453413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.454171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:48.454543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.456039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.486873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 376/744] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:48.488564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.543299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.610787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dda15-7d0f-4f0e-b34d-7f63af3b09dc None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.674923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.676031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.676530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.677099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.690693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.693890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.712613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24d06c-abd3-483f-a6a1-cc179d6f2e52 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:48.713270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.714449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 369/745] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:48.715789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.759196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.759830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.760158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.760462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.776007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:48.786163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.787966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 377/746] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:48.795909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.810487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.867216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe682a-d2ac-4bf4-b8e3-e6998a6079f9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.905692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.917226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.918005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.918438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.918936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.933044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.935024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.940093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052e17-ea88-43ce-8f4a-7102a1d70f55 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:48.954705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.956659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:48.958450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:48.958450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:48.958450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:48.960735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 370/747] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:48.961683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.985339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.985866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.986153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.986424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.987061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.988425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.998911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:49.001395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:49.002625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:49.005259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 378/748] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:49.006213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.090880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.128529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ccf7-666d-4963-b48a-bdfae62c2097 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.129856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3b4f-1a31-4b36-9187-b85d98a02060 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.169664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.170394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.170689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.170967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.171712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.172014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.173413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.183927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 379/749] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.184798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.196847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.197760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.198045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.198364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.199426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.199846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.201602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.219320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 371/750] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.220170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.285021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.334073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18557661-bad0-4e22-aa84-e699f9293ae1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.363066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4a11-7e27-48c8-b875-837ba80a7e5e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.378307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.385066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.386035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.386581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.387067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.394064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.394064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.394064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.414858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 380/751] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.414858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.415199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.415199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.415199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.415199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.415701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.415701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.415701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.425357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 372/752] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.427294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.554905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e5a92-00ef-49cd-a903-d147d549e6b4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.561220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331b08-9f58-4f88-82a3-0fb5c354236c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.564486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.605411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.606108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.606505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.607067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.607284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.607463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.607871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.608493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.609904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=a3b614a59a2e40c09dc87e99a717caf3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.610457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.612278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.621230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 373/753] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/role_assignments?user.id=a3b614a59a2e40c09dc87e99a717caf3&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.622342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.631821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.633978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:49.652561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:49.659347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:49.661789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:49.661789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:49.661789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:49.661789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 381/754] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:49.661789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.729675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.785451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979d21-7e25-4a4a-8f80-83d8bf2fded0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.813967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1d4a0-dd6a-4290-af06-c0ecfdfbbb2f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.828345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.828996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.829274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.829547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.837993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.839590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:49.862679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:49.863734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:49.864717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 374/755] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:49.866195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.881956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.883545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.884074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.884558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.885809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.886361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.888289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c46f4e-d98e-4052-88aa-75b9c2698081 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.895320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 382/756] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.897037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d6b5-dc08-4e58-b41c-ef6ac97d1fd4 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.897331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d6b5-dc08-4e58-b41c-ef6ac97d1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.897611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d6b5-dc08-4e58-b41c-ef6ac97d1fd4 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.899214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 383/757] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:49.899964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510d6b5-dc08-4e58-b41c-ef6ac97d1fd4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.919566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.037277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510d6b5-dc08-4e58-b41c-ef6ac97d1fd4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.045792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb717344-102f-4102-9187-0060bfdc4e18 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.057661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.093825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.094816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.095200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.095466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.096207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.096508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.098121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.104285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5352-7575-488e-adae-5a1018305809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.105709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5352-7575-488e-adae-5a1018305809 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.106567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5352-7575-488e-adae-5a1018305809 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.107385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5352-7575-488e-adae-5a1018305809 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.109312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5352-7575-488e-adae-5a1018305809 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.110110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5352-7575-488e-adae-5a1018305809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.111268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 375/758] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:50.112127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.112653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5352-7575-488e-adae-5a1018305809 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.137955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 384/759] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:50.139671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.140232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.140755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.162063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:50.163691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:50.246723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 385/760] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:50.246723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.287930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.306826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8086c-b095-4621-97cd-3237afe9888f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.390902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.391472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.391753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.392026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.392756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] RBAC: Adding query filter params (interface=public, service_id=42c8bfaebde64f34bbfa76f4b721c88b, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.393065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.394858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.409523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 376/761] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/endpoints?service_id=42c8bfaebde64f34bbfa76f4b721c88b&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:50.409523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.431531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.443591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cada2-06fe-4ba2-bfee-1897f1185df2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.509001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.509651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.509941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.510243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.532623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.535366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:50.553612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:50.559638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 386/762] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:50.561123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.567292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4266eaf-1eb0-496d-9ed5-709bba6ed6ae None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.613756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.614381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.614678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.614972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.641524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.701491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797f22-1337-4a91-b52b-eead45071186 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.728485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.731674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.744880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.745508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.745793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.746073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.746733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.748170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.754075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 387/763] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:50.758898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.806764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.901468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 377/764] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:50.902870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.960463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f2c1-c60d-44b8-ac4f-599ba6d54469 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.969631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.012000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.013127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.013896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.014384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] PATH_INFO: `/v3/services/network` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.015873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.017802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.027342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687af93-9c3d-474b-99aa-fe3c6f732dde None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:51.028579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:51.029984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.032993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 388/765] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:51.034639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.101459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.102581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.103079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.103525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.104709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.105200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.107423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.132917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 378/766] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:51.139223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.192985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.216485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5110d4a-1348-421e-b509-a2c656de41bc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.268138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.269581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.270195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.270980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.290392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.292734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:51.315033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:51.316245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:51.317612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:51.317612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:51.317612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.327036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 389/767] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:51.329258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.330085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9c80e-7162-4eea-82c9-894e0b4b4c0a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.344651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.372271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.374589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.375096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.375576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.376398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] RBAC: Adding query filter params (name=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.376843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.378636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.384358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 379/768] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:51.391099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.465446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312d44f-0fb7-468a-9022-d0612501a6f5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.507795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.508492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.508781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.509054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.512449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.517661 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97294f5-7a8c-4b39-abaa-a676e8a445ed None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.528240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.530035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:51.554731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:51.556031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:51.556967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:51.556967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:51.556967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.556967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 390/769] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:51.556967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.567351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.568281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.568719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.569170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.569983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.570488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.572302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.585961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 380/770] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:51.587291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.636903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.697327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9cf3-3a36-43fe-8e28-0a35730d4252 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.737119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aed2f-5ade-4f04-a063-19791a6664e0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.738214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.739071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.739528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.739957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.740937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] RBAC: Adding query filter params (type=network) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.741379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.742932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.749422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 391/771] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:51.750824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.789828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.790775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.791234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.791684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.792658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.793154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.795617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf35ef-520d-42ad-b358-ec0885b3d26b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.819013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 381/772] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:51.821041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def36fd-da4d-4c59-ba23-27ebeb652202 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.821542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def36fd-da4d-4c59-ba23-27ebeb652202 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.822055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def36fd-da4d-4c59-ba23-27ebeb652202 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.824538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 382/773] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:51.828492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def36fd-da4d-4c59-ba23-27ebeb652202 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.835649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.902743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9be67-fe19-4cf5-9f82-931f0e39ed5f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.944367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.945013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.945296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.945573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.957791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.960632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.965390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.974408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def36fd-da4d-4c59-ba23-27ebeb652202 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:51.981919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:51.984230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 392/774] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:51.985353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.045131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.046081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.046555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.047184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.048523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] RBAC: Adding query filter params (interface=public, service_id=054454cc84c24b38802fd8c78b2b178f, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:52.049051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.051686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1b461-0660-4139-925a-18d5eff91d37 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.062162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 383/775] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/endpoints?service_id=054454cc84c24b38802fd8c78b2b178f&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:52.064492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.064986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.065453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.096536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:52.098597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:52.103518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293479-afbe-434d-a0a5-d55de51c701e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.118972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.155681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.156764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.157269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.157754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.170681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.172717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:52.185352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:52.186613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 393/776] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:52.188408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.194997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 384/777] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:52.196088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.328752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.342157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bb5a-9489-4928-bf2a-126185b0d263 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.382314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.383111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.383466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.383812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.384515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.386067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.399792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 385/778] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:52.401502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.597495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3f9c-8582-4507-9d46-d4701134473f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.628782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86d87-bd25-4e1f-b892-e842af395771 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.658844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.663135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.663135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.663135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.663135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:52.663135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.663903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.673514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.674141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.674562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.674953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.675664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:52.676017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.678538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.680879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 394/779] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:52.681959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.699669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 386/780] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:52.700545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.726645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.741300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.871590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602f342-95fe-4e50-ab18-05d89dc60b31 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.876370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7015-c9e8-4b68-91a0-ad18847424ca None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.928933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.929640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.929976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.930535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.931330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=76a944cb3f8b4b639d9e4e5fceda0355) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:52.931775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.933285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.934752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.937140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.938624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.939657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.943084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 387/781] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/role_assignments?user.id=76a944cb3f8b4b639d9e4e5fceda0355&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:52.944829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.977493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:52.983707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:52.984768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:52.985626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:52.999840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 395/782] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:53.001142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.108272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.126902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65889014-a4f7-4128-8bae-18a90eb4826a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.143303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.156952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c87b80-0f62-4594-988c-2d00ae423378 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.175695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.176687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.177172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.178055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.180559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=0880178d7ea842afba2a0a4b8bd6d842, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.182472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ff4ed-50ea-439d-96ae-464f12fea070 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.216855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.217698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.217933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.218312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.219202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:53.219668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.221163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58589a-13a1-4d0c-91be-9332f86df325 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.232249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 396/783] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:53.234143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc194d-012e-45ba-a570-640d96948666 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.234476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc194d-012e-45ba-a570-640d96948666 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.234830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc194d-012e-45ba-a570-640d96948666 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.236530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 397/784] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:53.240236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc194d-012e-45ba-a570-640d96948666 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.263381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 388/785] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:52 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:53.272392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.273453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.274244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.312590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:53.315269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:53.458918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 389/786] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:53.486067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc194d-012e-45ba-a570-640d96948666 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.558200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.559417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.559918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.560414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.590887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.594194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:53.629241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:53.630703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:53.632222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:53.632222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:53.632222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:53.632222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 398/787] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:53.640193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.780689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.832656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd128a5b-0dd3-462e-a0b4-a4fa399724ce None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.892616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.893267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.893553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.893826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.894569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:53.894903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.896404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.915391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 390/788] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:53.921812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.939318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.947552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.063893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1ffe-ef83-457b-be5d-a0a6852bdb90 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.111409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.112071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.112355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.112626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.121440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.122937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.132758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f693315-991e-482a-a2cb-88eae83bcee5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:54.148119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:54.150465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:54.152851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 399/789] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:54.159186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.176012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.190159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.191632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.191910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.192181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.192780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.194343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:54.216926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:54.218094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:54.219483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 391/790] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:54.232601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.297698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.397744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63bd71-de0e-40bc-bd7f-5c3596959642 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.457338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7e854-373f-44ef-b464-f8cb138bb595 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.469506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.470195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.470483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.470781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.471534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:54.471840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.473366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.504285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 400/791] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:54.509793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.534273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.534944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.535226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.535501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.536155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:54.536461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.539929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.547488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 392/792] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:54.554074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.657859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.708259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d71858-8e79-44a9-a25b-9fa9b8fca49f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.754070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.754942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.755236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.755514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.756204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:54.756504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.757957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90927bdd-dec8-4e3d-8dfb-4642474631b0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.759933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd7f7f-d2f7-463c-913c-b2790388d38f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.766286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 393/793] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:54.768031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a749b2d-3396-41ce-b2b4-4650932fa076 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.768404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a749b2d-3396-41ce-b2b4-4650932fa076 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.768756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a749b2d-3396-41ce-b2b4-4650932fa076 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.770593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 394/794] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:54.781497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.781786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.782065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.797644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.807167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:54.808642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:54.812245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.812245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.812245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.812245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.827523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.831667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:54.870314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:54.872467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:54.874628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:54.874628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:54.874628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:54.880886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 401/795] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:54.880886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.948430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 395/796] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:54.950584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.052798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc50227-1e5d-4e2e-8ac8-b6f5ef892891 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.069930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.115471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.116516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.117003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.117492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.119044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.121603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad812b5-5ebb-42ad-94ca-2e530ce27263 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.128659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5313e-3f56-429c-829b-1ea89dbaf2da None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.167482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.168924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.169406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.169908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.170848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:55.171464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.173117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.187513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 396/797] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:55.193576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.270774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 402/798] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:55.308308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e480563-f351-42a8-af76-70ee38dadda8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.319163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.326213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c447dc-4c2f-44eb-a0b8-7a053e1d4aba None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.327721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c447dc-4c2f-44eb-a0b8-7a053e1d4aba None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.331415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c447dc-4c2f-44eb-a0b8-7a053e1d4aba None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.331415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 403/799] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:55.336582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.337147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.337854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.375385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:55.377507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:55.390978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.392055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.396354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.396949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.407875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.409739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:55.422744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:55.423745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:55.424636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:55.427010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 397/800] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:55.433605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.456279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.516105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 404/801] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:55.597805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.618217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cacc47-018f-4b9e-862f-563596d36276 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.678119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.678775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.679059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.679331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.680039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:55.680343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.681822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.696744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 398/802] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:55.705884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.763435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.779703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4300a4-fe85-48e1-96fc-07dadca9c626 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.880712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e6ed8-cdc5-487f-816a-98c44173f7d6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.884347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.885009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.885285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.885555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] PATH_INFO: `/v3/services/network` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.886160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.887589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:55.910788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:55.911873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:55.912728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 405/803] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:55.913555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.932227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.933196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.933478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.933755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/a3b614a59a2e40c09dc87e99a717caf3/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.937058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=a3b614a59a2e40c09dc87e99a717caf3, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.938543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.945301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.018024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 399/804] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:55 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/a3b614a59a2e40c09dc87e99a717caf3/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:35:56.019756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc2608-619d-4e6d-84ec-ada17658c460 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.020482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.070205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.070967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.071252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.071522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.085451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-d8a6ffca-e422-483e-9c6d-cd3b8b0b3176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:56.091974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:56.093566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:56.094942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 406/805] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:56.095842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c4c69-711a-4781-abfb-6ac89ba371cf None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.096093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c4c69-711a-4781-abfb-6ac89ba371cf None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.096252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c4c69-711a-4781-abfb-6ac89ba371cf None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.102353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 407/806] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:56.103565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c4c69-711a-4781-abfb-6ac89ba371cf None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.178350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.198886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be152a2b-4df9-4719-84c7-4ddafca205ba None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.228827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.256148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c4c69-711a-4781-abfb-6ac89ba371cf None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.261296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.261296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.261296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.261296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.261692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] RBAC: Adding query filter params (name=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:56.261763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.263348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7b66-579a-474b-8894-3be6da56eb85 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.269410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 400/807] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:56.270707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.271222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.271504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.293995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:56.295422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:56.312316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.313377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.313881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.314170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.315219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:56.315720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.318256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.334974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 408/808] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:56.336817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.372819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 401/809] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:56.378760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.605486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0ddd0-9932-40b9-889c-c03a92100d69 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.667806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.668876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.669386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.669879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.671033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] RBAC: Adding query filter params (type=network) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:56.671593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.673926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.698257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 409/810] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:56.705104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.719087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.740393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.746570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7b2f-57b0-4b15-8853-49daf985e12a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.800504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.801461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.801746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.802022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.816923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.820122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:56.838769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:56.839815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:56.840743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:56.840743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:56.840743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:56.846584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 402/811] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:56.847609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.880577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed885009-6c3e-4fc4-825c-82ddad7d9a82 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.926348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.927435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.927838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.928209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.929723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.931521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.978853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555af-cfe0-4650-a152-bceca771d235 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.020969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 410/812] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:57.020969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.054138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.075110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.075761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.076039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.076311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.104236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:57.115006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:57.116042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:57.117307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 403/813] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:57.125007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.182399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.249331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f8794-4adb-4155-ba65-069873307b27 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.290065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.291040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.291345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.291764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.292448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.292752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.294233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.298086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dd3eb-df2c-4e66-838c-6440cde0a434 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.309076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 411/814] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:57.315280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.344679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.345778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.346301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.346838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.347565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.347872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.349351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.361414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 404/815] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:57.368243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.438634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.483544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff583a7-c857-44ab-9e49-9d613cd4c724 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.509667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff79b1-ee57-47cf-bdf6-9e5b47e0604c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.527088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.545916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.563630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.564369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.565022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.565299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.565570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.566390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.577395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.578991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f50bae01-5188-459c-9329-efd96c980a72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:57.589030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:57.590073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:57.591389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 412/816] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:57.592007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267044b5-af06-42b0-b1e5-0fb769da5c6a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.592007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267044b5-af06-42b0-b1e5-0fb769da5c6a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.592679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267044b5-af06-42b0-b1e5-0fb769da5c6a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.597918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 413/817] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:57.597918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267044b5-af06-42b0-b1e5-0fb769da5c6a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-552cce63-03a8-4bda-aa96-af099b8d54c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:57.605892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:57.607020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:57.608020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:57.608020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:57.608020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:57.613962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 405/818] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:57.619395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.619826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.620154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.640313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:57.641558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:57.720983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267044b5-af06-42b0-b1e5-0fb769da5c6a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.762684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 406/819] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:57.762684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.773971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.774641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.774963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.775282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.776197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.776549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.778115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.799303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.800967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 414/820] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:57.806413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.870468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.903180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2438f25-64ce-4a08-95ba-769be2f784fd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.944834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e555f4-d026-4db9-988a-c79540a59ae1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.952980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.953729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.954162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.954632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.955602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.956482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.958296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.977273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 407/821] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:57.978434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.992052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.992934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.993222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.993638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.005897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.007473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:58.024581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:58.025824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:58.027035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:58.027035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:58.027035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:58.027035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 415/822] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:58.031877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.109499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.121333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceae8c-1d46-41a2-9ca1-20b8ec6edf1f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.160210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.164471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b21dc4-f1bc-48b7-9386-2c9cccdba0e9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.170630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.171320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.171601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.171877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.183042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.184600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:58.199535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:58.200759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 408/823] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:58.202351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.213214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.213843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.214134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.214459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.215779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.216096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.218483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.233731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 416/824] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:58.234515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.323484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98273-28d0-4f4a-8da9-d9e1007062cb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.338612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f056-925c-4c19-97ad-24374a4a67cc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.358943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.359578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.359860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.360156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.360745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.362134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-463c85ad-79cd-474b-864d-5b27ae26e10b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:58.370883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:58.371919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:58.373079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 409/825] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:58.374260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e1c7-d531-47ff-a1f4-156ae5a2e068 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.374260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e1c7-d531-47ff-a1f4-156ae5a2e068 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.374260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e1c7-d531-47ff-a1f4-156ae5a2e068 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.375251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 410/826] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:58.376479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9e1c7-d531-47ff-a1f4-156ae5a2e068 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.378896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.379604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.379884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.380155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.380910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.381219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.382706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.394112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.400646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 417/827] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:58.401477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.419648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.483177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9e1c7-d531-47ff-a1f4-156ae5a2e068 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.500878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97112-f0b9-479d-943d-2816752e21b6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.518200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.518861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.519142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.519414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.526293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.527770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.538152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.538781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.539061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.539330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2abd0a94-1f83-40e5-9bce-be1a9dd7aac5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:58.539443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:58.540311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:58.541233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:58.541679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.541679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.541823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 411/828] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:58.542011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.542247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.542513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.543868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.553374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 418/829] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:58.553374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.564441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:58.565753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:58.629156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.633698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 412/830] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:58.634575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.638396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.649851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91a24-a410-4254-b7be-9bff4312a622 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.685465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.686634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.686938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.687208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.737002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2e5b7-df0f-481e-a896-b15e912f2444 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.760331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.761826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.775168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.775784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.776061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.776323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.777028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.777328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.778794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.791222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 413/831] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:58.796832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.852287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.864091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.876318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 419/832] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:58.876318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.907979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d75d6-d532-474a-864c-69d3a0a9cac5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.943247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.943863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.944146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.944424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.945059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.945363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.946766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.951204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 414/833] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:58.951999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.976221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518121b-e6af-472e-943c-51b508749f83 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.014714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.015369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.015653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.015928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/76a944cb3f8b4b639d9e4e5fceda0355/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.033093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=76a944cb3f8b4b639d9e4e5fceda0355, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.034561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.071527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad29c-0c4f-4729-bdce-c37a09f2ca58 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.100479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.108641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.109289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.109572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.109847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.122905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.125051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:59.140201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:59.141387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:59.142291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:59.142291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:59.142291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:59.142291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 415/834] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:59.142291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.157299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 420/835] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:58 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/76a944cb3f8b4b639d9e4e5fceda0355/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:59.167783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.216079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.299563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641c1e4-8101-4d5b-8e85-9fd7cdb48e0f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.321802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be37b-17d9-4b4e-bb09-66ec7766baba None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.343707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.344355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.344611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.345214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.360926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.361532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.361807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.362076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.362742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:59.363100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.364155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:59.364388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:59.370914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:59.371925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:59.372812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:59.376070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 416/836] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:59.381688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 421/837] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:59.384970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.387628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.436707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.486062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.495584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e5ef-f288-4ab0-b320-07fdcff64299 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.504924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea5d7f-1b3f-48a8-bdad-c31cbd64b702 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.532374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.533013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.533297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.533574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.541228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.542778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:59.554546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:59.555688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:59.556737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:35:59.557665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 422/838] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:59.559842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.560460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.561296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.561574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.562249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:59.562643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.563705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.563982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.573824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 417/839] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:59.584002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.674599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a021a47-77d7-4a91-ad48-eb10e0b55ece None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.713190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d02e4b-5bf7-4b09-a63c-30724bb2ca52 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.714808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.715281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.715436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.715759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.716056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.717876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:59.718187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.719689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.737968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 423/840] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:59.738923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.752814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:59.757969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.759004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.759500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.760107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.774193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.776237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:59.795921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:59.797054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:59.797966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:59.797966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:59.797966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:35:59.800034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 418/841] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:59.809337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:59.866189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a2808-9db8-44e8-a1dd-d70c41835ae4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.904379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.905420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.905990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.906468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.907419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:59.907930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.909707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.923076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 424/842] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:59.926120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c72506-dc54-4635-9e52-011070ce809c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:59.969546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:59.970174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:59.970456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:59.970762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:59.971504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:59.971810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:59.973302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:59.989644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 419/843] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:59.995289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.090144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:00.118497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba79-9532-4765-bd0c-7dffde43c9d1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:00.161096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:00.161741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.162033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.162310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.172355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:00.174308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:00.188917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:00.189985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:00.191036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 425/844] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:00.195955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.212970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:00.303925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac906202-0935-4688-82aa-e3e4da37a340 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:00.343089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:00.343622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.343902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.344453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.344904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:00.345207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:00.346645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:00.364618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 420/845] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Aug 24 08:36:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:00.370287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.475353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aceabe-c9b1-48cf-b0d9-a39312e62dfc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:00.511610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:00.512469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.513042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.513483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.517137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:00.599045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:00.600500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:00.703016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 426/846] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:00.708838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.787972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:00.818986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529711-40df-40ca-a898-0700f534c154 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:00.858229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:00.858827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.859110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.859383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.870376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:00.871941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:00.887664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:00.889220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:00.890202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:00.890202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:00.890202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:00.890202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 421/847] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:00.899797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.936346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.015503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4575a3-9854-42b3-8bdb-24092aabc616 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.051171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.051800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.052084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.052358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.053014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:01.053321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.054716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5350-c224-41e2-977e-de90e1c3216d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.067520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 427/848] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:01.075803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.176846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc12b0-f636-4e12-a8ce-5fd2af40131e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.177225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc12b0-f636-4e12-a8ce-5fd2af40131e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.177508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc12b0-f636-4e12-a8ce-5fd2af40131e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.179286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 428/849] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:01.185141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d396206-0a51-4afe-8668-3affc9ff629f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.186012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d396206-0a51-4afe-8668-3affc9ff629f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.186256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d396206-0a51-4afe-8668-3affc9ff629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.199660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cca50-79a5-43f9-86d5-7843f1f6e2f0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.206973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1d396206-0a51-4afe-8668-3affc9ff629f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:01.208230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1d396206-0a51-4afe-8668-3affc9ff629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:01.230294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.243014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.243751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.244007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.244277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.252501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.254008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d9a63b14-4879-406a-845e-dd1d19cc16a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:01.267756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:01.268918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:01.270052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:01.272240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 422/850] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:01.273519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b338a-aa57-4ba5-ade3-700fbcdd032e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.273803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b338a-aa57-4ba5-ade3-700fbcdd032e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.274073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b338a-aa57-4ba5-ade3-700fbcdd032e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.276084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 423/851] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:01.280872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b338a-aa57-4ba5-ade3-700fbcdd032e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.294822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 429/852] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:01.294822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.294822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.294822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.316353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:01.318172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:01.339397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.400266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b338a-aa57-4ba5-ade3-700fbcdd032e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.406298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 430/853] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:01.448563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.449092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.449372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.449645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.450444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:01.450776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.452248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.466679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 424/854] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:01.472380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.575005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.597056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90492aeb-b8b8-4352-86d8-745d8606cd84 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.634976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.637772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.638491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.638872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.639215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.640025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=a3b614a59a2e40c09dc87e99a717caf3) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:01.640402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.641960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.648914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 431/855] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/role_assignments?user.id=a3b614a59a2e40c09dc87e99a717caf3&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:01.699021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b690e-d128-4f0e-9c42-eed0e2a1a2e2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.719492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.743393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.750077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.751104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.751765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.752283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.768440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:01.775494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:01.776536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:36:01.777402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:01.780556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 425/856] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:01.786606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.855504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d79331-cbb6-4550-8fea-a7b4443632b4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.902008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.902725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.903012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.903284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.903950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.905355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f31a3-5b40-4402-986c-03f592fd292b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.920612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974ddf-761e-4553-a624-6abf70f53eef None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.924001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.955898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.956431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.956713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.956989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.957682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:01.957987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.959427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.967943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 426/857] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:01.973266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.011108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 432/858] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:01 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:02.098523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc7a-d1b5-4a15-8b97-f024da7c2091 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.144894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.145550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.145831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.146106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.151992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf9b5-cfb8-40c7-82e7-2da59c966cd0 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.152517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf9b5-cfb8-40c7-82e7-2da59c966cd0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.152517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf9b5-cfb8-40c7-82e7-2da59c966cd0 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.154271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 433/859] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:02.161027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.161358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.161574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.162982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.164454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:02.180620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:02.181676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:02.183313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:02.183313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:02.183313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:02.183313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 427/860] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:02.186113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:02.187654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:02.187868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.208693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.278806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 434/861] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:02.307412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943eb9-a8b7-434a-89a8-51bed6c9a948 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.348641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.349426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.349812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.350189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.351033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:02.351438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.353000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.365926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 428/862] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:02.371601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.383089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.478405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f88ab-474d-4c2c-a733-26f09e4cb362 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.512985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.514160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.514441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.514737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.527678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.529205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:02.547511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:02.548522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:02.549934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 435/863] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:02.554524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.576487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.636947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.662699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb19e2-01d1-4423-8c1e-f80afaf58bb1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.674147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.682289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d881-ca07-46a1-a774-57ee7467b637 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.712966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.714499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.715137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.715598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.717077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:02.717560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.719011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.719586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.719695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.720060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.720400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.721057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.722543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:02.731948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:02.733002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:02.733920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:02.738165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 436/864] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:02.739477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.746306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 429/865] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:02.755727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.853187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe3696-a779-41f8-ad4a-e87aa0aa45b0 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.885754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.891612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.892290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.892505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.892778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:02.893415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:02.893728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:02.895125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:02.904185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa700d-cfbb-4d5b-8aa4-de034c25d12d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:02.904649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 437/866] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:02.905987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:02.947217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:02.955094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:02.956209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:02.957149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:02.957894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.015460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeac442-d641-4ee9-8ff8-a01a57f053f5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.056691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.057460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.057817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.058177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.058786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.058901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:03.059291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.060762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.061111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.074808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 438/867] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:03.195123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 430/868] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:02 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:03.202286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.272422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:03.317578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139081-6235-4589-bbf5-f3f9bb7f56cd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.353686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.354645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.355139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.355621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.366812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.368564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:03.384023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:03.385072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:03.385976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:03.385976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:03.385976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:03.387922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 439/869] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:03.394261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.427163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:03.503815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7451-cc59-4fe0-b9e2-0be754efac5c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.551284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.552362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.552849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.553326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.554213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:03.554895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.557031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e90d26-0931-4a8b-92db-3419129a5bf3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.571144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 431/870] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:03.577694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.699460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfadf3c-5c08-4406-8e30-81944502bfac None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.715312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:03.717374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c922-2c71-451d-a52d-7ce6d93ca6ed None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.717691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c922-2c71-451d-a52d-7ce6d93ca6ed None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.718325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c922-2c71-451d-a52d-7ce6d93ca6ed None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.719946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 432/871] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:03.726345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.726632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.731313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.745265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.746295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.746818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.747361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.750114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:03.751818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:03.756997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.758961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:03.776522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:03.777466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:03.778415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:03.780265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 440/872] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:03.786142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.848448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 433/873] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:03.906076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dbf6b-b9dc-4e7c-adfc-098e2147b08f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.942725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.943866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.944470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.944964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.946000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:03.946541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.948351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eddb4-d55b-412e-944d-3f71fb66a119 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.962281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 441/874] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:03.968654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.995317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.079143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a689d-ffa8-45e4-9e9e-7af4c49f2491 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.095286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b52-5902-4305-84fe-aae7e28d8317 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.095620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b52-5902-4305-84fe-aae7e28d8317 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.095831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b52-5902-4305-84fe-aae7e28d8317 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.097665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 442/875] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:04.103591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.103877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.104178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.121194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.122180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.122750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.123230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.124276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=76a944cb3f8b4b639d9e4e5fceda0355) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:04.124494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:04.124990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.125907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:04.126888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.139241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 434/876] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/role_assignments?user.id=76a944cb3f8b4b639d9e4e5fceda0355&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:04.149859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.168470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.224353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 443/877] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:04.269203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.283387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa3d1e-e8f1-468f-a7da-df2ba91cf619 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.321982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.322622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.322927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.323200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.323793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.325179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:04.334722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:04.335796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:04.336908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 435/878] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:04.342804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.456425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911dd28-0a6e-4871-a8eb-90ff2041f5db None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.501201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.502333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.502929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.503511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.504599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:04.505117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.506993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.513604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 444/879] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:04.518910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.527430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.578729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.652376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf173-1e42-4b0e-a74c-562bc81778d2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.693447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77857f-a928-4d5a-89c2-b32af868f26b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.708818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.709801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.710278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.710786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.713012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:04.713227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.714685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.722942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 436/880] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:04.747194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.748884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.749329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.749750 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.750497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.752204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:04.762610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:04.763947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:04.764857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:04.767328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 445/881] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:04.773653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.793646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.899399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74af6-5abf-48c3-8ef8-36d93565c574 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.937300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.938272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.938720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.939166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.939948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:04.940398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.941912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c234c-fda5-4331-b2a1-58e918086c9a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.947548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 437/882] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:04.953364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:05.044977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e5e-43e3-46b9-b778-b82d1f60321e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.045262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e5e-43e3-46b9-b778-b82d1f60321e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.045536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e5e-43e3-46b9-b778-b82d1f60321e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.050872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 438/883] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:05.052810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.053094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.053367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.054251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:05.064936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0b2ff-1beb-4eea-a994-e0d26230d19b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:05.073715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:05.075341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:05.108596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:05.109443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.109732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.110067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.111127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:05.111440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:05.113092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:05.119692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 446/884] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:05.125882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:05.162698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 439/885] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:05.253793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417707-9d08-4b0a-8b35-834d9cdff2cd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:05.289818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:05.290516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.290828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.291101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.291810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] RBAC: Adding query filter params (interface=public, service_id=be841f991173463bb70377a94edb69f3, region_id=RegionOne) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:05.292117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:05.293493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:05.302039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 447/886] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:36:05 2022] GET /identity/v3/endpoints?service_id=be841f991173463bb70377a94edb69f3&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:05.307499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:05.318855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:05.417911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1df6d9-4d45-4f94-a3d2-e6c9c199f64e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:05.441419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:05.454244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:05.454928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.455216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.455491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.456105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:05.457484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898b82-099a-420a-8491-d3d1a6f1c6b2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:05.462327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:05.463896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 440/887] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:05 2022] GET /identity/v3/services => generated 1011 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:36:05.559432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9ce4-a196-413b-8157-01095f2087dc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:05.599210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:05.600134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:05.600581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:05.601010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:05.601844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:05.603431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df8b0-e9a2-4645-8cec-63db30e4e113 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:05.683473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 448/888] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:05 2022] POST /identity/v3/services => generated 269 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:06.240026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623485-97e4-49c2-91cb-a7ac4e255f21 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.240551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623485-97e4-49c2-91cb-a7ac4e255f21 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.240551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623485-97e4-49c2-91cb-a7ac4e255f21 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.242192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 441/889] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:06.248197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a351-20e7-460a-94e4-039222f79e22 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.248490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a351-20e7-460a-94e4-039222f79e22 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.248769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a351-20e7-460a-94e4-039222f79e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.267497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9297a351-20e7-460a-94e4-039222f79e22 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:06.268695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9297a351-20e7-460a-94e4-039222f79e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:06.350688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 449/890] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:06.624721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623485-97e4-49c2-91cb-a7ac4e255f21 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:06.693257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14833-c93b-45b0-9417-d8d61b084a5a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.693588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14833-c93b-45b0-9417-d8d61b084a5a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.693875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14833-c93b-45b0-9417-d8d61b084a5a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.698685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 450/891] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:06.701171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.701459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.701771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.709130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:06.710124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:06.727471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:06.729076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:06.745153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623485-97e4-49c2-91cb-a7ac4e255f21 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:06.789047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:06.789773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.790056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.790946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.804693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:06.811174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:06.812199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:06.813048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:06.817821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 451/893] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:06.818002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 442/893] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:06.819040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:06.938579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812fbfa-3108-4326-91b4-b2e711b9949c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:06.975937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:06.976555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.976836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.977102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.977776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:06.978074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:06.979472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:06.980865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:06.989212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 452/894] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:06.995830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.108780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.115993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfbc5d-8594-433a-997a-8b358f667afb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.169845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.178454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.179566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.180072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.180557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.199758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.203139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.224470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcef81-6ffd-41d0-91f4-6b84b983eb21 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:07.230933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:07.231968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:07.232890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:07.232890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:07.232890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:07.233490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 443/895] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:07.239093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.262083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.262760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.263683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.266676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.266676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.266676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.350730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44f99-d744-4e12-88a1-1cefe4aaf5d3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.354872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 453/896] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:07 2022] POST /identity/v3/services => generated 241 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:07.409878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.410511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.410825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.411100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.411787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:07.412090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.413595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5470c2-ac5a-4d52-b6f0-444011de620f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.429184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 444/897] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:07.434828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.484714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e3900-a384-4d7f-a321-ff2a6e070dca None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.484993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e3900-a384-4d7f-a321-ff2a6e070dca None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.485264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e3900-a384-4d7f-a321-ff2a6e070dca None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.490993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 445/898] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:07.492395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.492677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.492948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.512350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:07.513710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:07.525993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.580555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.591306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f8fb-22e5-45c5-8cac-3c76d207a7ef None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.633213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 446/899] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:07.636951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.637591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.637921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.638192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.653447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.654927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3e8280f2-0942-4bdc-b0ff-00e98d8fbad2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:07.666983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:07.668051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:07.669646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 454/900] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:07.674694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.715299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750afa0-2773-48a0-8f40-cd21f027fd5c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.715613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750afa0-2773-48a0-8f40-cd21f027fd5c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.715925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750afa0-2773-48a0-8f40-cd21f027fd5c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.718680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 455/901] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:07.724427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.724427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.724645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.744187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:07.745465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:07.792467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfa30f-3d61-488f-8240-d66cb86b42e4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.830075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 456/902] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:07.833556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.834569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.835093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.835568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.836806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:07.837336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.839066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.860936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 447/903] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:07.866833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.904390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.919423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.939555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.981375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29f3b-46de-40e1-a68a-00bc10ae49e1 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.018542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.019393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.019727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.020101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.023752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b42a7-d096-4f0d-9c41-03cdc5e6607e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.061756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.062636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.063103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.063527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.064405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.066043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:08.076350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:08.077408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:36:08.078265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:08.081001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 448/904] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:08.086877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.099580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.101207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.148019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.202376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.206777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 457/905] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:08.207900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.211705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cca3-9ad3-4f68-8572-4b4c4744e8e4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.247635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.248825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.249105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.249378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.250037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:08.250370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.251830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.257158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 449/906] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:08.257798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.326811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551aa12-8b0f-4ce9-a7b6-6ca24f61129b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.368672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.369337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.369695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.369866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.370537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.371851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.373525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8e589-8808-4e98-a532-3a233abb9d44 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:08.381307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:08.382288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:08.383248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 458/907] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:08.384072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.400549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.414620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.415303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.417367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.417367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.426485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.427986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:08.443700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:08.444841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:08.445762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:08.445762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:08.445762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:08.445762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 450/908] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:08.445762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.459223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.497622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89f8d9-893d-4c41-a766-2edd4e176aa2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.538670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.539336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.539679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.539927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.540572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:08.540880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.542274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.547798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 459/909] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:08.548921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.564325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd19c-faad-4a5b-b615-d38092f8c96e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.604278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.605294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.605971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.606427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.607231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:08.607774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.610002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.620555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 451/910] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:08.620555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.637211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.665373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e7c34-cff4-4880-887c-1f6aee3da661 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.694680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.703290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.703933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.704216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.704576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.705137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:08.705548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.707266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.723683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 460/911] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:08.723683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.730445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c369-eb93-4407-9a9e-52422590e516 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.766960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.767569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.767847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.768251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.769021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.770437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.821438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872ce74-62fd-4c07-a64b-2e4c45c168e1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:08.857437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:08.858076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:08.858415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:08.858759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:08.859426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:08.859731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:08.861277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:08.863289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:08.867056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 452/912] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:08.871519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:08.873276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 461/913] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:08.884282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:09.009282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c9012-1e66-46f4-bc0f-51a293eee483 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.018756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d714a-6786-4443-ad96-9935bfd3dfb9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.037176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:09.052435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.053081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.053363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.053637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.062597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.065983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.069036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.069722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.070098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.070408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.071222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] RBAC: Adding query filter params (interface=public, service_id=371c4016f1134e08989de281e1d67562, region_id=RegionOne) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:09.071590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.073110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:09.077427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:09.078314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:09.079254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 453/914] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:09.084442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:09.086739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 462/915] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:36:08 2022] GET /identity/v3/endpoints?service_id=371c4016f1134e08989de281e1d67562&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:09.088136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:09.163531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:09.213272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b866-4610-4c12-9327-b7f61e6b25a2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.228713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd411f3-bff6-4b34-a8bc-56d5def426ac None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.252929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.253556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.253834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.254418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.255060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.256439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.266831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 463/916] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:09 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:36:09.280946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.281781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.282179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.282557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.283470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:09.283883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.285597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1d9a4-7548-4211-9887-52dacaa041df None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.301138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 454/917] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:09.307383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:09.429172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8418d-11db-435f-b163-38b9a3872235 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.437718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bcd2-0843-4dfb-8b58-c19f4433f14d None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.438314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bcd2-0843-4dfb-8b58-c19f4433f14d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.438314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bcd2-0843-4dfb-8b58-c19f4433f14d None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.439639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 455/918] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:09.444885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.445255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.445509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.457631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:09.469619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.470239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.470519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.470835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.472232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] RBAC: Adding query filter params (role.id=06db022d0ff24dafadb184483331f35f, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=76a944cb3f8b4b639d9e4e5fceda0355) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:09.472341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:09.472425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.473315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.473938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:09.479729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 464/919] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:09 2022] GET /identity/v3/role_assignments?user.id=76a944cb3f8b4b639d9e4e5fceda0355&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=06db022d0ff24dafadb184483331f35f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:09.526183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:09.558614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 456/920] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:09.831511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:09.937179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfcca-c630-4748-a572-c082f14524e3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:09.972157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:09.972802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.973087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.973365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.973956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:09.975353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:09.984971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:09.986002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:09.987029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 465/921] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:09.992087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.102386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c97435-d11f-4106-b40e-17e8fb95e343 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.139469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.140089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.140368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.140644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.141289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:10.141588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.143006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.148640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 457/922] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:10.155146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.214107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.267530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f516db-4bc8-4766-b7ca-4f8331e286a2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.307008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.308058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.308528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.308994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.309877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:10.310354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.311981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.318756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 466/923] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:10.330466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.383130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.432558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d89f0-1ff4-4232-94b2-f08419c65a39 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.470868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.471900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.471900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.472151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.472696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] RBAC: Adding query filter params (interface=public, service_id=79483d40ddfe45c18394d4edde7e3f45, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:10.472999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.474375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d713d-8632-4950-9442-0f514ddbdb55 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.480210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 458/924] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/endpoints?service_id=79483d40ddfe45c18394d4edde7e3f45&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:10.486557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.603395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31205386-6c58-4080-9ac0-c1742aad2c5e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.621715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.641946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.642533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.642843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.643136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.643831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.645502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b93b0-d6ca-4662-ba7c-bb59bb237389 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.653897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 467/925] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:36:10.950555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f151-2f46-4f71-ba33-aeb5b5af02df None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.951270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f151-2f46-4f71-ba33-aeb5b5af02df None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.951773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60f151-2f46-4f71-ba33-aeb5b5af02df None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.953661 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 459/926] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:10.959629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d944-5824-485a-8196-64a3109493b2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.960209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d944-5824-485a-8196-64a3109493b2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.960685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d944-5824-485a-8196-64a3109493b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.980247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1f33d944-5824-485a-8196-64a3109493b2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:10.982335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f33d944-5824-485a-8196-64a3109493b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:11.061928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 468/927] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:11.305303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60f594-4ec3-4522-853d-aae278f52848 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.305482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60f594-4ec3-4522-853d-aae278f52848 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.305767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60f594-4ec3-4522-853d-aae278f52848 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.307512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 460/928] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:11.313356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ca132-cabb-46ea-9e17-21366943713f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.314010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ca132-cabb-46ea-9e17-21366943713f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.314502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ca132-cabb-46ea-9e17-21366943713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.333390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-661ca132-cabb-46ea-9e17-21366943713f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:11.335070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-661ca132-cabb-46ea-9e17-21366943713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:11.341723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60f594-4ec3-4522-853d-aae278f52848 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.410330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 469/929] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:11.458431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60f594-4ec3-4522-853d-aae278f52848 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:11.460044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:11.495341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:11.495978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.496219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.496483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.497169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:11.498620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:11.501418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING keystone.server.flask.common [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:36:11.554161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.sql.core [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:36:11.554161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:36:11.554161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [parameters: {'id': 'f70aeeee11f041fbbe379f8fdc6e5d37', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:36:11.554161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93427) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-49e2870b-f14f-4d8a-90c0-6ba3f52a2b7f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f70aeeee11f041fbbe379f8fdc6e5d37.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f70aeeee11f041fbbe379f8fdc6e5d37. Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:36:11.572535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application result.read() Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 08:36:11.573390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application yield resource Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise self.value Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application session.commit() Aug 24 08:36:11.574370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:36:11.576012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:36:11.576942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application result.read() Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:36:11.577753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [parameters: {'id': 'f70aeeee11f041fbbe379f8fdc6e5d37', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.578637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:11.581086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f70aeeee11f041fbbe379f8fdc6e5d37. Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.582214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 461/930] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Aug 24 08:36:11 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:11.586983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ca132-cabb-46ea-9e17-21366943713f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.587437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2893d9-3717-4d68-b2b5-03a3ccc318f1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.587437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2893d9-3717-4d68-b2b5-03a3ccc318f1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.587437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2893d9-3717-4d68-b2b5-03a3ccc318f1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.596343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 462/931] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:11.602440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.602979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.603459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.624298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:11.626175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:11.634021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:11.693520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ca132-cabb-46ea-9e17-21366943713f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:11.701854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 463/932] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:11.703069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.737756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:11.738403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.738710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.738992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.747959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:11.748892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:11.761605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:11.763619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:11.764824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:11.768590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 470/933] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:11.769624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.824433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:11.843393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828ba50-2d7b-4f6d-9241-67951f2845fd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:11.883907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:11.884324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.884606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.884884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.885493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:11.887021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:11.890432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a383-e8db-4e0c-b8b5-6734452a6bdc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:11.893251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:11.898521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:11.899693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:11.901689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 464/934] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Aug 24 08:36:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:11.906536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.944884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:11.945584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.945872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.946174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.946880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:11.952179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:11.952179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:11.974973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 471/935] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:11.976144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.049416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d46219-dfc9-4e88-9b77-25162ba3795d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.088301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.089349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.089875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.090419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.091333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:12.091856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.093987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.105039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 465/936] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:12.106447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.127608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2af1-8a0e-46fe-bbe7-d052613bd4fa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.134061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:12.177479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.178160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.178443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.178775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.191957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:12.196820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:12.197803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:12.198687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 472/937] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:12.207181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.236190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:12.246137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a23d9e-3132-4274-94a4-15f6e3762a11 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.286417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.287170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.287535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.287911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.288619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:12.288992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.290958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.297492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 466/938] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:12.303860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.335768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aeead-8672-4b21-bd47-3a163cb919a4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.382285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.383569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.384077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.384835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.386631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:12.387181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.389381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.398126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:12.407159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 473/939] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:12.409117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.419066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b8f27-9d1e-4fbc-81a5-999040eaa3bb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.456997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.457553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.457830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.458100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.459150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.460287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.498862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:12.514908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391da9-49ae-4a85-b9ab-d8ad21feb261 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.555791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.556731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.557211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.557667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.569664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 467/940] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:12.585571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.587831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-751198cc-9c7d-4e79-8096-ae020a3e1309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:12.606252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:12.607358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:12.608299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:12.608299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:12.608299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:12.611088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 474/941] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:12.616875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.698235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c3c3e-8983-466a-811e-2e824de162d8 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.698605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c3c3e-8983-466a-811e-2e824de162d8 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.698821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c3c3e-8983-466a-811e-2e824de162d8 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.700527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 475/942] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:12.706398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.706711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.706993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.727623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:12.728873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:12.744119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca130-0887-4365-8297-146ce2772647 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.778550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:12.782928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782378-534d-4131-993e-f50caa61991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.783542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782378-534d-4131-993e-f50caa61991c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.783815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782378-534d-4131-993e-f50caa61991c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.784085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782378-534d-4131-993e-f50caa61991c None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.784736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782378-534d-4131-993e-f50caa61991c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:12.785034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782378-534d-4131-993e-f50caa61991c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.786416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782378-534d-4131-993e-f50caa61991c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.801193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 468/943] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:12.807695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782378-534d-4131-993e-f50caa61991c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:12.822891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 476/944] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:12.917046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782378-534d-4131-993e-f50caa61991c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:12.952996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:12.953544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:12.953821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:12.954088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:12.961864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:12.963364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:12.975626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:12.976666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:12.977663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 469/945] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:12.983090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.051610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.088056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66228e-c58a-4dc1-abfc-c5e79829b2de None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.096105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.133031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.133661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.134080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.134379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.135264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:13.135567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.138179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.158984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 477/946] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:36:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:13.164599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.174446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.231951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193db683-b752-4d71-8b0e-73158c068b1a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.269269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.270252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.270806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.271279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.272075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.273655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.273774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23227daa-9e2f-4fb0-871c-7f0e47cee438 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:13.283032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:13.284077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:13.284939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:13.292556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 470/947] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:13.293578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.299924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.311953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.313514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.313514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.313514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.324325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.326206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:13.341983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:13.343026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:13.343941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:13.343941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:13.343941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:13.346764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 478/948] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:13.349069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.411791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240de0a9-f548-48b2-8511-46948d0b93a6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.430352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.451280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.451523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e392c31-df6e-4519-9405-11d8d82ea922 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.452411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.453040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.453524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.454423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:13.454952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.456575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.468183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 471/949] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:13.469641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.488062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.488594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.488932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.489208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.489974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:13.490368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.491844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.506077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 479/950] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:13.511881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.557769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.588054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eba63e-49a3-4001-9f48-dd807b58240d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.628094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0a090-fcd2-4983-8cfb-c5bc4eb95ec6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.629207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.630175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.630803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.631308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.632167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:13.632670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.634284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.645271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 472/951] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:13.646645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.665141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.665784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.666060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.666338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.674643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.675881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:13.688737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:13.689792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:13.690867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:13.694854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 480/952] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:13.695818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.706742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.771164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea685cc-9413-4d38-a90f-a12bb61c44ad None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.799076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.805289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d279b-8cab-48be-b470-0e2952bdbb1d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.814427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.815438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.815949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.816436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.817363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.819026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f53036-7425-4416-aed5-98fc8580884b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.843931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.844543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.844823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.845103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.845772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:13.846076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.847533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.860314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 481/953] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:13.865760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.909839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 473/954] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:13.998615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377296e-7b8e-469e-a67c-32cc41c08c1b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:14.050368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14cf07-5253-4392-83ad-a533506d36eb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.050574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14cf07-5253-4392-83ad-a533506d36eb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.050688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:14.050996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14cf07-5253-4392-83ad-a533506d36eb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.051323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.051611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.051888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/76a944cb3f8b4b639d9e4e5fceda0355/roles/06db022d0ff24dafadb184483331f35f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.054345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=76a944cb3f8b4b639d9e4e5fceda0355, role_id=06db022d0ff24dafadb184483331f35f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:14.055979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:14.057790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 474/955] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:14.059285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.059568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.059837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.080711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:14.082645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:14.166047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 482/956] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:13 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/76a944cb3f8b4b639d9e4e5fceda0355/roles/06db022d0ff24dafadb184483331f35f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:36:14.170691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 475/957] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:14.447073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:14.463122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:14.553120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467e393-1f58-494b-ba4a-ccd32c5cd57f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:14.588977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:14.589636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.589928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.590239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.591044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:14.592475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:14.595038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING keystone.server.flask.common [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:36:14.634342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.sql.core [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:36:14.634342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:36:14.634342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [parameters: {'user_id': '2c1e759412bb45f7bf4b352a5456f51c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:36:14.634342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ffe5a0e0-9948-403d-bf51-d2e6b634df7d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result.read() Aug 24 08:36:14.641770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:36:14.642742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application yield resource Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise self.value Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application session.commit() Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:36:14.645063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:36:14.645999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:36:14.647962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application result.read() Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:36:14.650386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [parameters: {'user_id': '2c1e759412bb45f7bf4b352a5456f51c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:14.652476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 08:36:14.653976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:14.657813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 483/958] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Aug 24 08:36:14 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:14.658765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:14.659305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e54f2-9877-4c42-b1fd-2c2f7998409e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.659305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e54f2-9877-4c42-b1fd-2c2f7998409e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.659305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e54f2-9877-4c42-b1fd-2c2f7998409e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.662891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 484/959] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:14.668319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.668612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.668892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.690306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:14.692027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:14.693142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:14.774155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 485/960] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:14.783430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182397b3-2c67-47f7-8717-9ccb8e5f3520 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:14.821681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:14.822325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.822607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.822909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.831288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:14.832763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:14.845376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:14.846383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:14.847397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 476/961] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:14.852550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:14.857441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:14.956986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac32c1-133b-454b-8378-4a72f5bcb8a4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:14.993017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:14.993675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:14.993966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:14.994243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:14.995049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:14.995356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:14.997481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:15.033319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 486/962] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Aug 24 08:36:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:15.096034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:15.202054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:15.212651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ebe3-feb4-45ab-be04-c6a1c2416190 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:15.250454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:15.251220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.251566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.251906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.252573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:15.254019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:15.263842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:15.265242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:15.266295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:15.267366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 477/963] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:15.273303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:15.342709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:15.382023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec769b-3a21-4e49-aaf1-14ed2f1806bf None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:15.416147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:15.416900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.417260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.417609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.418899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:15.419278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:15.420715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17178d8-fa1c-4659-901d-4ff480ed2794 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:15.430443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 487/964] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:15.431789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:15.544504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb121c-394e-4738-b28d-5cbe49111639 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:15.578834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:15.579446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.579729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.580025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.580685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:15.580988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:15.582423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a7e09-6ee2-4c78-a272-dfd7fda6de28 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:15.587598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 478/965] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:15.922803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6a9a-09d8-45f6-9c09-130d9b3cde8e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.923346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6a9a-09d8-45f6-9c09-130d9b3cde8e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.923346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6a9a-09d8-45f6-9c09-130d9b3cde8e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.924854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 488/966] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:15.930260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd696dae-8ede-46c4-bb46-e4a744facfc5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.930986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd696dae-8ede-46c4-bb46-e4a744facfc5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.930986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd696dae-8ede-46c4-bb46-e4a744facfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.950150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cd696dae-8ede-46c4-bb46-e4a744facfc5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:15.951380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cd696dae-8ede-46c4-bb46-e4a744facfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:15.966431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.022474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 479/967] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:16.190492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b0a1-d9dc-4e3f-bee2-35dc188147c2 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.190841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b0a1-d9dc-4e3f-bee2-35dc188147c2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.191008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b0a1-d9dc-4e3f-bee2-35dc188147c2 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.192335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 489/968] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:16.197839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.198130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.198405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.217488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:16.218710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:16.286354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76b0a1-d9dc-4e3f-bee2-35dc188147c2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.289398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 480/969] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:16.390327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76b0a1-d9dc-4e3f-bee2-35dc188147c2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.416823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.426754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.433073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.433073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.433073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.433073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.433073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:16.439688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:16.440841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:16.441717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 490/970] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:16.446778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.538049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.561929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa4de4-4cae-4770-a83f-8994e90834ff None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.562468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.598034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.598640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.598940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.599208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.599857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:16.600130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.601495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.606917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 481/971] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:16.612310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.677518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12f5b3-507d-45a8-bd71-3ac29d84e4fd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.714967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.715537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.715816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.716713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.728596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:16.731649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539dd5-ba97-4a66-a417-884c06a8ce18 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:16.735893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:16.738842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:16.739788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 491/972] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:16.743133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.774872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.775519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.775798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.776074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.777004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:16.777120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.779023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.784727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 482/973] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:16.790753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.800317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.812153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.873694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71755d80-1a74-490d-a68e-d57bf1f642ea None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.913145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34a0a4-bf3a-480d-8eed-f3f5985bb9c7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.921892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.922516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.922831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.923391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.923785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:16.924079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.925466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.934159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 492/974] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:16.939635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.953374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.953999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.954262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.954537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.955312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] RBAC: Adding query filter params (interface=admin, service_id=79483d40ddfe45c18394d4edde7e3f45, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:16.955620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.957033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.962901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 483/975] 10.222.0.64 () {56 vars in 1249 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/endpoints?service_id=79483d40ddfe45c18394d4edde7e3f45&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:16.969166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.048128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.050607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85dbb-6c10-44cd-8bf5-2d6e63cf4e9d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.066533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.080449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53030-ab20-4266-a13d-55c243b43429 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.089143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.090335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.090843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.091118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.102289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.103949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.118422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a088-635a-4190-972b-182b8bde009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:17.118809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:17.119797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:17.120754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:17.120754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:17.120754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:17.120930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a088-635a-4190-972b-182b8bde009c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.120930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a088-635a-4190-972b-182b8bde009c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.120930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a088-635a-4190-972b-182b8bde009c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.121168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 493/976] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:17.123149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a088-635a-4190-972b-182b8bde009c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.124693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a088-635a-4190-972b-182b8bde009c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.125550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.138697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 484/977] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:36:17.239213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db90d4-de00-4627-880a-a6693efa3a9c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.276054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.276593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.278031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.278031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.278601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:17.279053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.281146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.295160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 494/978] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:17.300763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a088-635a-4190-972b-182b8bde009c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.404376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a088-635a-4190-972b-182b8bde009c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.438673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.439101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.439379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.439653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.441602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.447927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.449913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5172f152-cf28-4f34-998d-4ba6ddb193c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:17.462053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:17.463312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:36:17.464519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:17.465415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 485/979] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:36:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:17.470598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.497524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.514634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0969b8-5f1b-4125-a652-93e701dc6e06 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.514968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0969b8-5f1b-4125-a652-93e701dc6e06 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.515233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0969b8-5f1b-4125-a652-93e701dc6e06 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.516737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 486/980] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:17.522354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.522639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.522940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.546494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:17.547840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:17.596278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e34e7-5893-4584-9baa-f775953edbc5 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.625210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 487/981] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:17.632879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.633518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.633800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.634074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.634849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:17.635155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.636620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.655250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 495/982] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Aug 24 08:36:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:17.660929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.765746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c551-c892-4eb2-be28-b19ccfb3e36f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:17.790018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.801848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:17.802483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:17.805274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:17.805274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:17.875250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:17.876732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:17.890618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:17.979175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:17.988635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 488/983] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:17.989713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:18.003826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9b59a-4288-4c5e-ad2b-9dc7a433c784 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.016643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:18.040045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.040655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.041005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.041281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.041940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.043460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.095640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456334b-a1be-4834-a989-b365f0abc9a1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.130682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 496/984] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:17 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:18.140835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.141587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.141885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.142142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.155695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.157196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:18.174273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:18.175532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:18.176533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:18.176533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:18.176533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:18.177238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 489/985] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:18.181449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:18.294521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ececa8-b92e-4b00-8172-501a4d0a4607 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.331232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.332011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.332386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.332751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.333500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:18.333891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.335369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.350730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 497/986] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:18.353651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:18.425468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:18.456478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210c86-6cb7-4041-8b56-b48e4eead139 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.491646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.492471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.492838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.493205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.501134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.502718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:18.514996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:18.516194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:18.517382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:18.526160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 490/987] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:18.527343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:18.530692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:18.634451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec65da-db88-45d3-a9ec-f0f66ca54f45 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.670770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.671380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.671664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.671934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.673131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:18.673131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.674365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35117d-1ea7-4da8-a184-ec0b74db0c7f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.692769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 498/988] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:18.694837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:18.800680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68f31c-0354-4a81-97eb-9d6372111ae7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:18.830069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:18.835502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:18.836392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.842407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.842407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.842407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] RBAC: Adding query filter params (role.id=06db022d0ff24dafadb184483331f35f, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=76a944cb3f8b4b639d9e4e5fceda0355) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:18.842407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:18.842407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1c3f6-08c9-4afa-afcb-c47fa0dd1539 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:18.853724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 491/989] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:18 2022] GET /identity/v3/role_assignments?user.id=76a944cb3f8b4b639d9e4e5fceda0355&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=06db022d0ff24dafadb184483331f35f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:19.061558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4aaa-b6e6-4e2a-bd92-92b592281b61 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.062226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4aaa-b6e6-4e2a-bd92-92b592281b61 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.062226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4aaa-b6e6-4e2a-bd92-92b592281b61 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.063627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 499/990] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:19.068861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.069302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.069397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.087874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:19.089079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:19.161394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 492/991] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:19.430461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4aaa-b6e6-4e2a-bd92-92b592281b61 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.498416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.549011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4aaa-b6e6-4e2a-bd92-92b592281b61 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.594854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.595778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.596225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.596663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.597401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.599003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:19.608733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:19.610007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:36:19.611135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:19.612553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 500/992] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:19.618688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.728904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684c2d-ea7d-4e7a-a622-8520cf3b80f1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.757302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.767159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.767771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.768048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.768850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.768948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:19.769230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.770609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.775741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 493/993] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:19.781940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.889002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d62a-f254-47da-9ab3-1d335e6eb21e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.916739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.927543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.928037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.929025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.929025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.929271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:19.929517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.930989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdfb8f-a49f-4ff5-8344-627b27c6fcb4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.936412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 501/994] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:19.942368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:20.052018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d88ce-6d18-42e2-aa1b-4b0e6e46a9d2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:20.068833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6232-9164-4837-8830-114cb278996c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.069361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6232-9164-4837-8830-114cb278996c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.069361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6232-9164-4837-8830-114cb278996c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.070759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 502/995] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:20.076137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.076285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.076435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.090504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:20.091160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.091440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.091710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.092383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:20.093782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:20.096588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:20.097785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:20.168623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 503/996] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:20.260544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 494/997] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:20.440118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:20.575388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc2ac-ff38-4909-b845-456b9d264f08 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:20.583647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:20.605583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:20.606319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.606691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.607061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] PATH_INFO: `/v3/services/compute` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.607716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:20.609149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-684288bb-7471-4a8e-aced-eace0b186f6f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:20.618086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:20.619229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:20.620118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:20.621862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 504/998] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:20.625882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:20.678156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:20.824841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3fde2-acb0-4dd4-9174-fc8e6eca9c0e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.825224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3fde2-acb0-4dd4-9174-fc8e6eca9c0e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.825547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3fde2-acb0-4dd4-9174-fc8e6eca9c0e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.827073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 505/999] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:20.832315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.832596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.832870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.842043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514349a-ce43-455a-bdf8-9aef8b7042d3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:20.850836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:20.852037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:20.877651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:20.878371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.878773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.879047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.879672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:20.879968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:20.881461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:20.891670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 495/1000] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:20.892697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:20.926224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 506/1001] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:21.001639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ec5e-2852-40e1-ad2b-8b79e84f6f42 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.015618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:21.035335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.035939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.036219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.036529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.037119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:21.037421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.038826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.044255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 496/1002] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:21.050034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:21.154238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6970dc-b0a7-40af-826e-7a1857ebfc4d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.165363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:21.189882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.190501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.190831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.191109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.191919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] RBAC: Adding query filter params (interface=public, service_id=a19f4c7a6a364b7bb22ab2bbbc25d4d5, region_id=RegionOne) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:21.192188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.193569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.197828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:21.203461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 507/1003] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:36:21 2022] GET /identity/v3/endpoints?service_id=a19f4c7a6a364b7bb22ab2bbbc25d4d5&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:21.204411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:21.306912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c644fce-ad77-46e8-b4d7-54dcb1de6259 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.313932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd14cf0-24c7-486e-a746-c8c0c9116bca None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.344332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.344971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.345255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.345534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] PATH_INFO: `/v3/services/image` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.349461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.350069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.350328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.350438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.350596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.351229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.351756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.352804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.360536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 508/1004] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:21 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:21.376948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:21.378089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:21.379747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 497/1005] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Aug 24 08:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:21.384411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:21.429649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:21.492056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9e947-4da9-4a3a-9561-dd8f754b25da None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.535181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.535790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.536068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.536346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.536982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] RBAC: Adding query filter params (name=image) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:21.537280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.538646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73586d5-8b64-4ce5-aa9a-175cca92038e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.545392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 509/1006] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:21.551196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:21.598102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:21.657096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490f14-46f1-4531-8337-9346b36af998 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:21.692567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:21.693093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:21.693368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:21.693638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:21.694281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] RBAC: Adding query filter params (type=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:21.694581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:21.696072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718421ff-5023-446b-8e68-d18c087ee019 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:21.701182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 498/1007] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:22.202007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c51-1083-4893-81d6-eda821b8e23e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.202582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c51-1083-4893-81d6-eda821b8e23e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.202582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688c51-1083-4893-81d6-eda821b8e23e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.204120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 510/1008] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:22.209681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.210150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.210254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.229535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:22.231473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:22.302697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 499/1009] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:22.565793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688c51-1083-4893-81d6-eda821b8e23e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:22.675129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688c51-1083-4893-81d6-eda821b8e23e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:22.677289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:22.713409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:22.714044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.715007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.715284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.718510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:22.720092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d50c32-fd9d-4c6e-9914-054424c92f07 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:22.721904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 511/1010] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:22.727961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:22.776138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:22.847623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dc220-a899-400a-93ce-a97a9a58dc33 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:22.887132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:22.887749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.888027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.888294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.888939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:22.890415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1dad7-5937-48c7-bdd8-e4f350469ca5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.086847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 500/1011] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:22 2022] POST /identity/v3/projects => generated 311 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:23.299595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f5acc-daef-4c19-af80-1b2400c15286 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.300154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f5acc-daef-4c19-af80-1b2400c15286 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.300154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f5acc-daef-4c19-af80-1b2400c15286 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.301667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 512/1012] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:23.307526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72db8-12e2-417d-91a1-7db1935aaaed None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.308565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72db8-12e2-417d-91a1-7db1935aaaed None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.308830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72db8-12e2-417d-91a1-7db1935aaaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.328170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c4c72db8-12e2-417d-91a1-7db1935aaaed None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:23.329383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c4c72db8-12e2-417d-91a1-7db1935aaaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:23.403021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 501/1013] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:23.649406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd71d2-2d04-4e42-bed8-99b99295bc56 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.649773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd71d2-2d04-4e42-bed8-99b99295bc56 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.649986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd71d2-2d04-4e42-bed8-99b99295bc56 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.652079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 513/1014] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:23.661217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.661217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.661217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.671586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd71d2-2d04-4e42-bed8-99b99295bc56 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:23.678205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:23.679431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:23.695100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:23.755121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 502/1015] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:23.781282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd71d2-2d04-4e42-bed8-99b99295bc56 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:23.820959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:23.821592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.821870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.822139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.822738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:23.824096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:23.832746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:23.833753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:23.835921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 514/1016] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:23.840769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:23.943987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2200c3d-621b-44aa-82d5-8820708e7b5e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:23.978637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:23.979270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.979544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.979842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.980480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:23.980778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:23.982153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.991802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 503/1017] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:23.993192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:24.020702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:24.030392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:24.039852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:24.101404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ab985-dac4-4a3d-b3e4-83b74cf88041 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:24.127768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d7cd-83a9-4010-b994-214d4271b080 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:24.137657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:24.138271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:24.138553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:24.138878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:24.139517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:24.139814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:24.141181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:24.150985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 515/1018] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Aug 24 08:36:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:24.152791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:24.181199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:24.181815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:24.182093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:24.182363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:24.183099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:24.184475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef576dc6-1154-4767-9bb3-e971c34a8a7c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:24.253355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:24.257043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b7ed-288a-4675-bd2b-213dc44c9ec9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:24.290875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:24.291088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:24.291350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:24.291631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:24.292316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:24.293693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08af56a-a281-4815-8751-f09f903268f7 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:24.297800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 504/1019] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:24 2022] POST /identity/v3/services => generated 242 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:24.411950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 516/1020] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:25.076046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22fabd-e801-4a46-b102-8776be0d47cb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.076576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22fabd-e801-4a46-b102-8776be0d47cb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.076576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22fabd-e801-4a46-b102-8776be0d47cb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.078114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 505/1021] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:25.083248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.083545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.083822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.102291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:25.103529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:25.178578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 517/1022] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:25.446939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22fabd-e801-4a46-b102-8776be0d47cb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:25.527047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:25.561982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22fabd-e801-4a46-b102-8776be0d47cb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:25.600184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:25.600841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.601113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.601382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.602603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:25.604086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feca60-951f-4fc1-9c6a-e5c7ece5f31b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:25.605911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 506/1023] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:25.611749 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:25.694829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:25.715507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402015a5-dd80-4371-a091-c4af0b4a4bd3 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:25.756702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:25.757320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.757605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.757911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.758540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:25.760145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7777373-8656-4334-9d81-409381f2ec8a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:25.866275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 518/1024] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:25 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:26.237647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa95d-6560-4e73-a259-19a164aa8653 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.238260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa95d-6560-4e73-a259-19a164aa8653 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.238260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa95d-6560-4e73-a259-19a164aa8653 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.239709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 507/1025] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:26.245118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8892e-71f3-444d-b692-e0b3c8e86af8 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.245407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8892e-71f3-444d-b692-e0b3c8e86af8 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.245682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8892e-71f3-444d-b692-e0b3c8e86af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.264345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-72a8892e-71f3-444d-b692-e0b3c8e86af8 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:26.265525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-72a8892e-71f3-444d-b692-e0b3c8e86af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:26.337704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 519/1026] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:26.432497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20623528-094b-4616-9324-113d9b275d71 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.432860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20623528-094b-4616-9324-113d9b275d71 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.433038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20623528-094b-4616-9324-113d9b275d71 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.434545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 508/1027] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:26.439909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.440250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.441160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.459238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:26.461055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:26.526817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.535403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 520/1028] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:26.609805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20623528-094b-4616-9324-113d9b275d71 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.721609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20623528-094b-4616-9324-113d9b275d71 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.763281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.764361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.764856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.765328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] PATH_INFO: `/v3/services/image` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.766192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.768826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:26.785337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:26.786463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:26.787394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:26.795076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 509/1029] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:26.796381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.808793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.812712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.908200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e4ef-f2fd-4b57-b152-6ac7bddb9ac4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.931923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09afe1-ee74-4279-a93a-89d59eb9c963 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.946028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.946663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.946962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.947243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.947906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] RBAC: Adding query filter params (name=image) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:26.948579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.949624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.958694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 521/1030] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:26.960921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.975022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.975662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.975942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.976277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.989022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:26.993902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:26.995538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:26.996464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 510/1031] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:27.004495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.051145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.083285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.099361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8509b-ee8b-41b4-b7dc-fe551f55783f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.127854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59b14-845f-42af-90c3-26576d9ace67 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.137947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.138565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.138890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.139161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.139837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] RBAC: Adding query filter params (type=image) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.140102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.141601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.148646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 522/1032] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:27.154591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.171654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.172271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.172556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.172830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.173501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.173802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.175362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.186902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 511/1033] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:27.192123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.274397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbe5-6130-404d-a96b-34f2484e4d3e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.299719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a930d0-4888-46cf-b676-97031676e949 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.301838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.312328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.312942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.315859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.315859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.315859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] RBAC: Adding query filter params (interface=public, service_id=539061f6dcdb433aa04489728bcd39f9, region_id=RegionOne) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.315859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.315859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.325614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 523/1034] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/endpoints?service_id=539061f6dcdb433aa04489728bcd39f9&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:27.326690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.333183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.344276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.344916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.345197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.345470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.357407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.358917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:27.373475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:27.374721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:27.375660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:27.375660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:27.375660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:27.379703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 512/1035] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:27.380574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.437454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5a9c-d04a-4a21-9abb-7b1b9908f283 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.482970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.483643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.483864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.484140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.484758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.486123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.488712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b10-7354-4692-99ba-73ea3b86de2a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.493276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 524/1036] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:36:27.529591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.530225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.530511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.530858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.531526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.531879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.533344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.553246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 513/1037] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:27.554462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.615352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.669607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0b7e-d657-4de4-8539-93ba04283d32 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.706608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.707267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.707550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.707822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.717133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.718813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f2818898-1309-461a-87b8-051cef35c6f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:27.731479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:27.732526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:27.733523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 525/1038] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:27.740664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.783685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:27.845261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b298c-7823-48d4-9efc-fad1cfd95024 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.879406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.880025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.880308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.880584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.881328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.881759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.883112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.883263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd3fe5-5b2c-431c-8db4-bdeb61c57d8c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.883695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd3fe5-5b2c-431c-8db4-bdeb61c57d8c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.883773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd3fe5-5b2c-431c-8db4-bdeb61c57d8c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.885233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 526/1039] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:27.891005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.891288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.891558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.903658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 514/1040] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:27.909195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:27.910436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:27.911650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:27.990934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 527/1041] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:28.013882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.020665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1883-e03f-4f94-b3ca-9f69579160dd None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.057510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.058136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.058402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.058698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.130131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.131639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.230332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.237439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 515/1042] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:36:28.243190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.258360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.287728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.346041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052848d-1621-4330-98a7-62a85decd3c2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.369446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb61afb-0f78-43ea-baaf-2c3055058ae7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.381448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.382020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.382286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.382560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.393623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.395112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.408540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.409172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.409448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.409721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:28.409888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:28.411163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:28.412650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:28.412650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:28.412650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:28.412650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 528/1043] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:28.416400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.426041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:28.430840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:28.432013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:28.433468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 516/1044] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:28.438772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.499648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.517401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.525533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238a99-475a-4686-b5ec-bb5acded323f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.556375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962ff0b-4bd4-402f-8951-877eb1d04da3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.565986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.567268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.567754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.568524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.569651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:28.570134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.572263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.588971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 529/1045] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:28.593122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.593730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.594013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.594116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.594277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.595018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:28.595323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.596758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.607226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 517/1046] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:28.612994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.704501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b9864-d4a7-4773-b340-f73c89a7c14a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.721974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b576454-b6d6-401f-8cab-d65bebc0f655 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.738145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.741658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.742503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.743000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.743554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.751069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.752680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.755497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.756134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.757529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.757529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] PATH_INFO: `/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:28.766277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:28.767402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:28.768450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:28.770449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] RBAC: Authorizing `identity:get_user(user_id=00ce7a1cc64c4771b58c4456a8bc25bd)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.770581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 530/1047] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:28.772003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.774689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 518/1048] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:28.779600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.780752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.797007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:28.884448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30d0a7-7499-4c22-93ad-0faf175f09be None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.888073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c94b0e-0634-4406-94c0-d3d85bbc394e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:28.921604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.922634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.922773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:28.923266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:28.923509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.923614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:28.923792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.924256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:28.925208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:28.925531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:28.928099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:28.946612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 531/1049] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:28.949060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:28.979755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.005659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.007156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.035451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.066893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab1d1a-756c-418d-867d-8f2bc68a2243 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:29.110168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:29.110811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.111090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.111359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.112084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] RBAC: Adding query filter params (role.id=06db022d0ff24dafadb184483331f35f, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:29.112371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 519/1050] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:29.117301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.117301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.118271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.120432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 532/1051] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:28 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=06db022d0ff24dafadb184483331f35f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:29.209286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.234548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87acfee-7fb6-4659-8b0f-24356d4ae365 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:29.274199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:29.274867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.275134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.275401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.276543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.277985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648c4d6-645a-4cdb-badb-223b1e6c2325 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.279807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 520/1052] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:29.285698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.391569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bdd20-eaef-44d8-b701-2806f5018dae None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:29.427911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:29.428522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.428797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.429069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.429820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=2ae911173c334b3a8d8c580e5d774f73, user.id=00ce7a1cc64c4771b58c4456a8bc25bd) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:29.430126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.431591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9103-3406-495d-8605-9f5e69b02647 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.446693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 533/1053] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/role_assignments?user.id=00ce7a1cc64c4771b58c4456a8bc25bd&scope.project.id=2ae911173c334b3a8d8c580e5d774f73&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:29.456322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e966-4caf-4c9f-a82c-9c0e30b67de2 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.456864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e966-4caf-4c9f-a82c-9c0e30b67de2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.456864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e966-4caf-4c9f-a82c-9c0e30b67de2 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.458313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 521/1054] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:29.464715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.465022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.465373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.471952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.485370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:29.486507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:29.571479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 534/1055] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:29.836894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e966-4caf-4c9f-a82c-9c0e30b67de2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.962369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e966-4caf-4c9f-a82c-9c0e30b67de2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:30.002289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:30.002980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:30.003261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:30.003530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] PATH_INFO: `/v3/services/image` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:30.004131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:30.005563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:30.015566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:30.016658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:30.017536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 522/1056] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:30.023277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:30.136234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cfae1-2e18-4cba-9cd5-6e5f61cf2d47 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:30.169800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:30.175405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:30.176022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:30.176311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:30.176576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:30.177259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] RBAC: Adding query filter params (name=image) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:30.177566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:30.179009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:30.186208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 535/1057] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:30.191204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:30.203720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:30.297397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32184fa8-e3af-40ff-b197-411042f3d296 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:30.333196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:30.333844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:30.334098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:30.334368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:30.335301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] RBAC: Adding query filter params (type=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:30.335604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:30.337004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3617aa-9da0-4b8f-8dca-03c848601578 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:30.347300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 523/1058] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:30.348440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:30.451935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6e90c-0ce9-47d7-9767-15383c95b31a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:30.488350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:30.488961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:30.489245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:30.489519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:30.490263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:30.491567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67551-0629-4605-8b1f-09efe3de9caa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:30.585487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 536/1059] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:36:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:31.028909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215a57-9df8-47d4-833b-823da50c8172 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.029435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215a57-9df8-47d4-833b-823da50c8172 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.029435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215a57-9df8-47d4-833b-823da50c8172 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.030876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 524/1060] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:31.036220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23fdd-106b-4251-8a4c-c7a97d8163bc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.036644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23fdd-106b-4251-8a4c-c7a97d8163bc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.036851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23fdd-106b-4251-8a4c-c7a97d8163bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.055311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-45b23fdd-106b-4251-8a4c-c7a97d8163bc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:31.056508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-45b23fdd-106b-4251-8a4c-c7a97d8163bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:31.106863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:31.132464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 537/1061] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:31.356220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2270-395f-4ae7-8f76-7fdd1a161860 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.356467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2270-395f-4ae7-8f76-7fdd1a161860 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.356808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2270-395f-4ae7-8f76-7fdd1a161860 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.358222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 525/1062] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:31.363476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.363764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.364035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.381885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:31.383082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:31.399908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d2270-395f-4ae7-8f76-7fdd1a161860 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.453166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 538/1063] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:31.474136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:31.504543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d2270-395f-4ae7-8f76-7fdd1a161860 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:31.546372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:31.547062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.547343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.547617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.558887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:31.563743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:31.564720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:31.570366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 526/1064] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:31.571792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.676610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2efcd1-16e4-473b-92b9-c553ce32d761 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:31.687284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:31.714770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:31.715637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.716073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.716498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.717320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:31.717803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:31.719352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:31.724070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.733269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 539/1065] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:31.734795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.831789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d727-b8b4-46a7-8b63-435b67ebcbfe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:31.851712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc928a7-f4e2-4be9-8f52-d8546f374f59 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:31.861437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:31.867141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:31.867784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.868065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.868331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.879627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:31.884499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:31.891147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 527/1066] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:31.892249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.897573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:31.898398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.898880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.899321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.911918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:31.913624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:31.929556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:31.930919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:31.932932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:31.932932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:31.932932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:31.932932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 540/1067] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:31.938839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:31.968569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.005598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf047d-7e15-49f0-b110-471d040c5f20 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.042936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.043622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.043853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.044084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.044749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:32.045053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.046437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.047341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472846c6-89e9-4517-98ba-0944bc233c81 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.058690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 528/1068] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:32.061824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.087664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.088322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.088624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.088940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.089553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:32.089859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.091285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.108414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 541/1069] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:32.109512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.125602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.173020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076317b2-a7e6-42c2-b674-359c52fddfd4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.200176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.211320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.212214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.212651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.212923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] PATH_INFO: `/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.214278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] RBAC: Authorizing `identity:get_user(user_id=00ce7a1cc64c4771b58c4456a8bc25bd)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.215754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.217502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 529/1070] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:32.218284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74c8-4842-4edb-8886-9a90aa30c35d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.223089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.258056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.258719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.259006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.259439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.266673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.268250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:32.280238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:32.287752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 542/1071] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:32.288950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.348068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a302d15-1dd1-4155-815a-76a14845c152 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.372732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.385262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.385896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.389124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.389124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.389124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.389124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.395828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 530/1072] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:32.396928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.411734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aceca9d-b36e-4c02-a439-dd6548097e18 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.451314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.452284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.452567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.452855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.453570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:32.453869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.455352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.469616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.473250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 543/1073] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:32.479558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.513013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d6465-f2d8-4394-b788-885f149288fe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.557289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.558115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.558463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.558950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73/users/00ce7a1cc64c4771b58c4456a8bc25bd/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.561066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ae911173c334b3a8d8c580e5d774f73, user_id=00ce7a1cc64c4771b58c4456a8bc25bd, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.562558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254eb6d-4bda-4307-8d46-d98396030265 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.591419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c773-0f50-461c-af14-ada8999b7943 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.626867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.627495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.627780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.628059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.638921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.640399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-8b2819bc-1d36-4155-b15f-704f1314f514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:32.656667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:32.657899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:32.658991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:32.658991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:32.658991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:32.658991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 544/1074] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:32.659249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b70-bf73-4893-b690-99d1f3f6c760 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.659335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b70-bf73-4893-b690-99d1f3f6c760 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.659633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b70-bf73-4893-b690-99d1f3f6c760 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.660847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 545/1075] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:32.663611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f9b70-bf73-4893-b690-99d1f3f6c760 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.682637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 531/1076] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:32 2022] PUT /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73/users/00ce7a1cc64c4771b58c4456a8bc25bd/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:36:32.684752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.685255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.685723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.720161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:32.722012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:32.730928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.788966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f9b70-bf73-4893-b690-99d1f3f6c760 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.789180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.812485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 532/1077] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:32.830533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.831351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.831697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.832048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.832789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:32.833225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.835117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.851236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 546/1078] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:32.858254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.966388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5167a3-f70a-44b9-99f6-fc46d2a89767 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.001183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.001740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.002019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.002288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.009274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.011299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:33.022807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:33.023947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:33.025225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 533/1079] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:33.030337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:33.081860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:33.128947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:33.133910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d8fad-39f6-4d12-b662-e5405496ba03 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.149644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:33.171910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.172515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.172796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.173065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.173756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:33.174081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.175557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.190695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 547/1080] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:33.196513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:33.200005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434864fa-1f4d-4846-acd7-f290cfcfe6bb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.235121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.235734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.236008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.236278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.302707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8481-f39d-4a00-8a88-2389ad0af12e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.310889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.312365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.339495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.340226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.340517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.340795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/06db022d0ff24dafadb184483331f35f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.343010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=0880178d7ea842afba2a0a4b8bd6d842, role_id=06db022d0ff24dafadb184483331f35f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.344447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.354841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:33.411627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 534/1081] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:33.417353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:33.439029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 548/1082] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:33 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/0880178d7ea842afba2a0a4b8bd6d842/roles/06db022d0ff24dafadb184483331f35f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:36:33.545979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0570e-aced-434f-af85-f5f63207e8d8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.589052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.589637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.589916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.590179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.599206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:33.601441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.603022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9bd5364e-7597-47e9-af85-e74219d7ef6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:33.618253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:33.619978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:33.620935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:33.620935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:33.620935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:33.621579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 535/1083] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:33.626435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:33.660312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:33.727949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0d1b4-6a85-4620-b0a0-30d349c87565 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:33.762694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:33.767776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7b9f7-b8fd-492a-91e4-d52e0614866a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:33.779142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 549/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:34.676740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dff17-065b-4fe1-bff6-2facb884036c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:34.686216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dff17-065b-4fe1-bff6-2facb884036c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:34.686216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dff17-065b-4fe1-bff6-2facb884036c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:34.686216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 536/1085] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:34.686495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:34.686495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:34.686495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:34.702355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:34.703566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:34.767816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 550/1086] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:35.034767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dff17-065b-4fe1-bff6-2facb884036c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.146225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dff17-065b-4fe1-bff6-2facb884036c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.184636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.185262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.185540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.185812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.196121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:35.200925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:35.202050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:35.202959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:35.206827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 537/1087] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:35.208666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.234809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.322417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc0dff-a970-4f79-9663-575c570c1a8e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.359001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.359636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.359922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.360201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.360886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:35.361189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.362580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ea91-f57d-4340-8c33-976285fcd2fa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.372275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 551/1088] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:35.378417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.386033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17afaca-8643-4287-af80-c5813835c728 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.386353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17afaca-8643-4287-af80-c5813835c728 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.386629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17afaca-8643-4287-af80-c5813835c728 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.388011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 552/1089] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:35.393313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.393606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.393886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.412759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:35.413981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:35.443256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.483461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 553/1090] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:35.489592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e32cd-0e5c-4a05-b90d-1e25f2f4883b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.523794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.524429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.524709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.524981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] PATH_INFO: `/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.526642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] RBAC: Authorizing `identity:get_user(user_id=00ce7a1cc64c4771b58c4456a8bc25bd)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.528111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.529711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 538/1091] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/users/00ce7a1cc64c4771b58c4456a8bc25bd => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:35.535675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.551704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.633466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f684c-3a7b-488b-a021-13d28be071a4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.666310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.666958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.672606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.672606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.733747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.735244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.749375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.763299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.833196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 554/1092] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:35.839938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.862488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338c04-9b5f-44a5-8b4e-71027b604c09 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.882574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.897325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.897883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.898167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.898435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.909599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:35.914367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:35.915511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:35.917057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 539/1093] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:35.922128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.942943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee655-e2e2-40e5-a939-549805616a7a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.957094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:35.976755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.977387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.977672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.977950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.979580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.981003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.982740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 555/1094] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:35.990711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.022692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d471-a0e8-424f-a45c-1e23a8085e2b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.057268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.057876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.058180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.058454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.059208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:36.059513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.060913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.069060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 540/1095] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:36.074458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.089798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ee43-7748-428a-8230-638f5a54f1a8 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.118125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:36.122907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.123512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.123791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.124062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.124781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=2ae911173c334b3a8d8c580e5d774f73, user.id=00ce7a1cc64c4771b58c4456a8bc25bd) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:36.125088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.126515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.132284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 556/1096] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:35 2022] GET /identity/v3/role_assignments?user.id=00ce7a1cc64c4771b58c4456a8bc25bd&scope.project.id=2ae911173c334b3a8d8c580e5d774f73&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:36.181000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35408dc-5899-4432-a0b6-0e81e3487fad None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.218212 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.218896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.219172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.219438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.231635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.233114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:36.248986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:36.255499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:36.257041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:36.257041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:36.257041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:36.257041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 541/1097] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:36:36.257343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.306582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:36.363589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccf71-134c-40ff-beec-f5fb48dc6f64 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.397411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.398014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.398294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.398565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.399392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:36.399695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.401539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.413672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 557/1098] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:36.419790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.512937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:36.519498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455c5fd-4ada-4edf-9f9a-f96abd02af06 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.553502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.554129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.554404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.554699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.562015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.563489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:36:36.574899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:36.575969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:36.576977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 542/1099] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Aug 24 08:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:36.581907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.682223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104358f-854f-4384-96ea-3dccff5c0ef1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.689626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:36.716602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.717208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.717489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.717762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.718493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:36.718834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.720268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:36.736982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 558/1100] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Aug 24 08:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:36.742542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:36.841877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe11ab0-9bec-44b4-bd9a-4ae1383c65c7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:36.874328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:36.874961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:36.875239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:36.875501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:36.899567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:36.942152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:36.943685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.039385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 543/1101] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:37.045216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.143171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d166f-91cc-4b8f-814c-c846ed13489f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.179169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.179806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.180091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.180368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.191409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.192877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:37.208406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:37.209719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:36:37.211264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:37.211264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:36:37.211264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:37.211264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 559/1102] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:37.215700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.287047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.340557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcac99-94bc-4576-a16d-2ad008190de7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.375694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.376312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.376581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.376857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.377524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:37.377824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.379265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.391991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 544/1103] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:37.397611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.416300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.501877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ceafe-c5da-47a7-a2f9-135dfa6339cf None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.541207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.541856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.542132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.542401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.549425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.550897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:36:37.562748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:37.563896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:36:37.565020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:36:37.566805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 560/1104] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:37.569750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.677074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf424f3-23cd-46ac-91b7-173971b2ebe6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.692446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.713447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.715918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.715918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.715918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.715918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:37.715918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.717151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe1c46-70ae-47f7-8d1a-d54f82717ef9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.729699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 545/1105] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:37.735747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.816666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.839674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7729211-6fd4-4893-bd3b-a8c3ca547f34 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.875072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.875791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.876074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.876340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.877062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] RBAC: Adding query filter params (role.id=06db022d0ff24dafadb184483331f35f, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=0880178d7ea842afba2a0a4b8bd6d842) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:37.877367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.878804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841e0df-1209-4c75-84ba-3df927ca7da4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.884508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 561/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/role_assignments?user.id=0880178d7ea842afba2a0a4b8bd6d842&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=06db022d0ff24dafadb184483331f35f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:38.085313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91d37-dbc3-4c51-820c-c838ea98b262 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:38.085819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91d37-dbc3-4c51-820c-c838ea98b262 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:38.085819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91d37-dbc3-4c51-820c-c838ea98b262 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:38.087207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 546/1107] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:38.093877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:38.094167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:38.094440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:38.114587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:38.115868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:38.194456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 562/1108] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:38.463918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91d37-dbc3-4c51-820c-c838ea98b262 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:38.582236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91d37-dbc3-4c51-820c-c838ea98b262 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:38.619955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:38.620593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:38.620877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:38.621152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:38.622285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:38.623847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755e5bb-2c42-41d3-b305-8c329154c7af None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:38.625435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 547/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:38.631435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:38.702781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:38.732558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfe336-e07c-40f0-9f56-5d1409de61b6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:38.767929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:38.768577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:38.768859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:38.769130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:38.769776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:38.771501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e1cd7-039e-4014-85a6-1561b351e75a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:38.865939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 563/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:38 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:40.861629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a6de-e422-4ea8-8199-9a6085e4e251 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.862146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a6de-e422-4ea8-8199-9a6085e4e251 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.862146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373a6de-e422-4ea8-8199-9a6085e4e251 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.863687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 548/1111] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:40.869180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.869600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.869706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.887521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:40.888706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:40.954332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 564/1112] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:41.226065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373a6de-e422-4ea8-8199-9a6085e4e251 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.273281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.329617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373a6de-e422-4ea8-8199-9a6085e4e251 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.363231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.363854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.364133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.364401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] PATH_INFO: `/v3/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.365801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] RBAC: Authorizing `identity:get_role(role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.367210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.368579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 549/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:41.374531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.431684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.477670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd3b62-e6b6-492e-9239-4da099071fdc None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.513200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.513846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.514128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.514400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] PATH_INFO: `/v3/users/889f8c894d934c48acc27fd05c24b64f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.531776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] RBAC: Authorizing `identity:get_user(user_id=889f8c894d934c48acc27fd05c24b64f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.533237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.534999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 565/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/users/889f8c894d934c48acc27fd05c24b64f => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:41.541143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.640296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaa45a-0103-4ee4-b71b-72a050e3d70f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.650798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.672999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.673604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.673883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.674155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.742482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.744381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.843365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 550/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:36:41.849362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.948706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de01ad-0c1b-492d-9d77-16636ce0c14b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.962629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.982282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.982942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.983231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.983509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.984623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.986053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5579f5-7c6a-40e5-b5f1-dbe2ae3088e6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.987725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 566/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:41.994646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:42.006114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:42.092223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd60792-c87c-43cd-ae0e-fcbf8449eac1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:42.125909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:42.126508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:42.126887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:42.127161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:42.127887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=2ae911173c334b3a8d8c580e5d774f73, user.id=889f8c894d934c48acc27fd05c24b64f) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:42.128194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:42.129622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5bc21-baf5-4e7a-a7f9-4d7c44d0a1f3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:42.134894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 551/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/role_assignments?user.id=889f8c894d934c48acc27fd05c24b64f&scope.project.id=2ae911173c334b3a8d8c580e5d774f73&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:44.016766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11942e6b-0341-4daf-b80e-4a8526298f8d None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.017378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11942e6b-0341-4daf-b80e-4a8526298f8d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.017378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11942e6b-0341-4daf-b80e-4a8526298f8d None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.018643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 567/1118] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:44.023821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.024252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.024405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.042021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:44.043255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:44.111842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 552/1119] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:44.378694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11942e6b-0341-4daf-b80e-4a8526298f8d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.475687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11942e6b-0341-4daf-b80e-4a8526298f8d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.512762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.513386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.513674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.513941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] PATH_INFO: `/v3/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.515397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] RBAC: Authorizing `identity:get_role(role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.516794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.518183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 568/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:44 2022] GET /identity/v3/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:44.524884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.528187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:44.631100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06383-9a8a-4bf8-ac9b-ec525afb6461 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.664348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.665533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.665807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.666073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] PATH_INFO: `/v3/users/889f8c894d934c48acc27fd05c24b64f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.667217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] RBAC: Authorizing `identity:get_user(user_id=889f8c894d934c48acc27fd05c24b64f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.668646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.670231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 553/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:44 2022] GET /identity/v3/users/889f8c894d934c48acc27fd05c24b64f => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:44.676098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.724125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:44.771729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78b8e-93c7-43f8-bc7a-5b86c2e501e0 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.804869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.805520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.805798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.806064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.807186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.808613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf394f-95f8-43d7-b12d-07cee31ffe5f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.810219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 569/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:44 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:44.816248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.884820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:44.913078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af4168-49b6-4b3d-b3f3-6acda9678519 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.946787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.947445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.947722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.947993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73/users/889f8c894d934c48acc27fd05c24b64f/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.950116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] RBAC: Authorizing `identity:create_grant(project_id=2ae911173c334b3a8d8c580e5d774f73, user_id=889f8c894d934c48acc27fd05c24b64f, role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.951577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298976-dc64-45be-a84d-ee520c2b083e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:45.051343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 554/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:44 2022] PUT /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73/users/889f8c894d934c48acc27fd05c24b64f/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:36:46.934810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341a6fa-560f-4b11-bb42-15542d5e8873 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:46.935405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341a6fa-560f-4b11-bb42-15542d5e8873 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:46.935405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341a6fa-560f-4b11-bb42-15542d5e8873 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:46.936686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 570/1124] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:46.941924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:46.942362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:46.942482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:46.961841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:46.963089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:47.030279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 555/1125] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:47.298944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341a6fa-560f-4b11-bb42-15542d5e8873 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.397754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341a6fa-560f-4b11-bb42-15542d5e8873 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.415561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.433297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.433929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.434210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.434482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] PATH_INFO: `/v3/roles/c78c0c380cda4c08ab3942220b3b6402` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.435908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] RBAC: Authorizing `identity:get_role(role_id=c78c0c380cda4c08ab3942220b3b6402)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.437278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.438595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 571/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/roles/c78c0c380cda4c08ab3942220b3b6402 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:47.444501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.504845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.549173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ff4b7-936a-400a-8f55-a9c7a637e719 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.594006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.594635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.594934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.595207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] PATH_INFO: `/v3/users/889f8c894d934c48acc27fd05c24b64f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.596332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] RBAC: Authorizing `identity:get_user(user_id=889f8c894d934c48acc27fd05c24b64f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.597771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.599415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 556/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/users/889f8c894d934c48acc27fd05c24b64f => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:47.606080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.701989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddccbd-697e-485e-93e3-51ed1d444e92 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.737241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.737851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.738134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.738406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.797787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.811655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.813065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.919637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 572/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:47.927113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:48.030330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76604317-26af-4d00-96a2-e359733ada22 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:48.058032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:48.063853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:48.064426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:48.064704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:48.064976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] PATH_INFO: `/v3/projects/2ae911173c334b3a8d8c580e5d774f73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:48.066094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] RBAC: Authorizing `identity:get_project(project_id=2ae911173c334b3a8d8c580e5d774f73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:48.067574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6493a90-6740-45f7-8635-417103981d1f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:48.069222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 557/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/projects/2ae911173c334b3a8d8c580e5d774f73 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:48.076600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:48.165998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:48.180210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355007-441d-4ac0-ac8d-f3d07d3ff3fc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:48.217489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:48.218089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:48.218367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:48.218635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:48.219385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] RBAC: Adding query filter params (role.id=c78c0c380cda4c08ab3942220b3b6402, scope.project.id=2ae911173c334b3a8d8c580e5d774f73, user.id=889f8c894d934c48acc27fd05c24b64f) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:48.219690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:48.221102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6434f0-bb60-48d7-b5f8-8ffadda91a7e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:48.227142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 573/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:48 2022] GET /identity/v3/role_assignments?user.id=889f8c894d934c48acc27fd05c24b64f&scope.project.id=2ae911173c334b3a8d8c580e5d774f73&role.id=c78c0c380cda4c08ab3942220b3b6402 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:50.223820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5ca4-25b5-4973-a47e-cb581f1a2699 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:50.224400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5ca4-25b5-4973-a47e-cb581f1a2699 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:50.224400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e5ca4-25b5-4973-a47e-cb581f1a2699 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:50.225710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 558/1131] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:50.231552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:50.231986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:50.232085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:50.250160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:50.251383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:50.315150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 574/1132] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:50.585259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e5ca4-25b5-4973-a47e-cb581f1a2699 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:50.694349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e5ca4-25b5-4973-a47e-cb581f1a2699 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:50.731513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:50.732142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:50.732421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:50.732717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:50.733830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:50.735310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfaa83a-fe99-41ee-b144-8a4c50dc4beb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:50.736895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 559/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:50.742807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:50.821473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:50.840656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1b139-734a-45d9-9302-73a0f08795a8 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:50.874521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:50.875165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:50.875443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:50.875718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:50.876369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:50.877797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c99fb-815b-4b73-bff8-cb74f81aebd1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:50.975115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 575/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:50 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:52.957448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef5d40-0adf-4024-be64-34dc3eb693e7 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:52.958011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef5d40-0adf-4024-be64-34dc3eb693e7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:52.958011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef5d40-0adf-4024-be64-34dc3eb693e7 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:52.959625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 560/1135] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:52.965098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:52.965508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:52.965712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:52.983969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:52.985181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:53.052642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 576/1136] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:53.330562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef5d40-0adf-4024-be64-34dc3eb693e7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:53.398167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:53.435316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef5d40-0adf-4024-be64-34dc3eb693e7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:53.469078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:53.469705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:53.469990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:53.470263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:53.471380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:53.472815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5a13e-0c98-4301-bebf-ad3e08d7c823 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:53.474394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 561/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:53.480689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:53.540874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:53.578795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f89830-acd3-44ec-9781-224503eaf370 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:53.613639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:53.614616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:53.614935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:53.615211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:53.615858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:53.617285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af177ba6-233a-4f62-a6cd-b460b1cc2b34 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:53.739222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 577/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:53 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:55.738349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d1ad-dc6b-45a6-89ad-ab77ff12b3dd None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:55.738915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d1ad-dc6b-45a6-89ad-ab77ff12b3dd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:55.738915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d1ad-dc6b-45a6-89ad-ab77ff12b3dd None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:55.740286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 562/1139] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:55.745721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:55.746117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:55.746375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:55.764638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:55.765843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:55.833446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 578/1140] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:56.099008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d1ad-dc6b-45a6-89ad-ab77ff12b3dd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:56.206908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d1ad-dc6b-45a6-89ad-ab77ff12b3dd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:56.240271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:56.240896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:56.241178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:56.241478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:56.243488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:56.252319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:56.260696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:56.261930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:56.263109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 563/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:56.265208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:56.314399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:56.368188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9dd70-5d6c-4ee5-b991-1e1684e2fddf None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:56.403183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:56.403788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:56.404069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:56.404340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:56.405010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:56.405337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:56.406687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:56.414567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 579/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:56.420575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:56.519608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89ca1-f6ed-4d49-91cc-3b913ed040a2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:56.558000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:56.558631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:56.558934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:56.559229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] PATH_INFO: `/v3/users/6084e84345db4ac7b7257332d4015324` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:56.573039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] RBAC: Authorizing `identity:get_user(user_id=6084e84345db4ac7b7257332d4015324)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:56.574494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:56.576182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 564/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:56 2022] GET /identity/v3/users/6084e84345db4ac7b7257332d4015324 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:56.582178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:56.680096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c47d4-c8fa-496b-9737-9d88eb454957 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:56.690603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:56.723059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:56.723671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:56.723952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:56.724223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:56.790392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:56.791878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:56.886736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 580/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:56.892850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:56.944986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:56.991108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083116f2-88dd-4583-aed2-1c741f836c22 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:57.024513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:57.025136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:57.025417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:57.025689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] PATH_INFO: `/v3/projects/efec666a75724e2d86c3b57071f259a1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:57.026846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] RBAC: Authorizing `identity:get_project(project_id=efec666a75724e2d86c3b57071f259a1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:57.028282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638bfa8-6447-4a56-8eeb-0bb93269be7a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:57.029923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 565/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:56 2022] GET /identity/v3/projects/efec666a75724e2d86c3b57071f259a1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:57.036115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:57.047981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:57.134095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe550a-0904-4ea7-b35b-2e684f7d774a None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:57.168272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:57.168870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:57.169155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:57.169431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:57.170152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=efec666a75724e2d86c3b57071f259a1, user.id=6084e84345db4ac7b7257332d4015324) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:57.170457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:57.171921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0150337-e08c-40ad-9ad7-300d599f3154 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:57.177569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 581/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:57 2022] GET /identity/v3/role_assignments?user.id=6084e84345db4ac7b7257332d4015324&scope.project.id=efec666a75724e2d86c3b57071f259a1&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:59.101068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374c61-f02e-4575-88d1-38ba77df579d None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:59.101642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374c61-f02e-4575-88d1-38ba77df579d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:59.101642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374c61-f02e-4575-88d1-38ba77df579d None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:59.103000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 566/1147] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:59.108465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:59.108916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:59.109053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:59.127499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:59.128748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:59.195396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 582/1148] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:59.464278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374c61-f02e-4575-88d1-38ba77df579d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:59.568445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374c61-f02e-4575-88d1-38ba77df579d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:59.589055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:59.603344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:59.603961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:59.604256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:59.604512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:59.614556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:59.619462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:59.620703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:36:59.621989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 567/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:59.626817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:59.629985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:59.726301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb46d44-770b-4de8-9e13-65ebeb06327e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:59.759525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:59.760136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:59.760419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:59.760712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:59.761352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:59.761652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:59.763047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:59.771057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 583/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:59.777593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:59.877300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830927d-e365-4aae-830c-8e23c406737b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:59.911330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:59.911951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:59.912228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:59.912501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] PATH_INFO: `/v3/users/6084e84345db4ac7b7257332d4015324` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:59.913760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] RBAC: Authorizing `identity:get_user(user_id=6084e84345db4ac7b7257332d4015324)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:59.915238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:59.916836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 568/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:59 2022] GET /identity/v3/users/6084e84345db4ac7b7257332d4015324 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:59.922810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:59.986509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:00.019435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6275-39ea-4434-9941-60377a7a107c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:00.051912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:00.052542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:00.052824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:00.053095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] PATH_INFO: `/v3/projects/efec666a75724e2d86c3b57071f259a1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:00.054218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] RBAC: Authorizing `identity:get_project(project_id=efec666a75724e2d86c3b57071f259a1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:00.055717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70911797-7f97-4fd5-8bde-9194251e39e6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:00.057334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 584/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:59 2022] GET /identity/v3/projects/efec666a75724e2d86c3b57071f259a1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:00.063392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:00.112176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:00.162278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77124-f352-4683-b618-b7a8385efa74 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:00.196378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:00.197067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:00.197345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:00.197617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] PATH_INFO: `/v3/projects/efec666a75724e2d86c3b57071f259a1/users/6084e84345db4ac7b7257332d4015324/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:00.199546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] RBAC: Authorizing `identity:create_grant(project_id=efec666a75724e2d86c3b57071f259a1, user_id=6084e84345db4ac7b7257332d4015324, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:00.200973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e8bf8-938c-4969-870d-1bbe8b8e16dd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:00.316158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 569/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:37:00 2022] PUT /identity/v3/projects/efec666a75724e2d86c3b57071f259a1/users/6084e84345db4ac7b7257332d4015324/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:37:02.267855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00860193-c9df-4a98-8673-124bc4356e22 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:02.268440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00860193-c9df-4a98-8673-124bc4356e22 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:02.268440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00860193-c9df-4a98-8673-124bc4356e22 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:02.269741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 585/1154] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:02.275113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:02.275562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:02.275666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:02.293732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:02.294959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:02.361166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 570/1155] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:02.637725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00860193-c9df-4a98-8673-124bc4356e22 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:02.740082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00860193-c9df-4a98-8673-124bc4356e22 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:02.773538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:02.774091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:02.774374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:02.774671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:02.776516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:02.786005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:37:02.791524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:37:02.792657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:37:02.793729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 586/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:37:02.799102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:02.838361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:02.898370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3631d8-2f48-4b97-a861-c302f505dc47 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:02.930828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:02.931439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:02.931920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:02.932352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:02.933575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:02.933871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:02.935271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:02.943116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 571/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:02.949715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:03.046209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c19c-6158-4439-8bf9-46f5062ada56 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:03.079012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:03.079636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:03.079914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:03.080182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] PATH_INFO: `/v3/users/6084e84345db4ac7b7257332d4015324` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:03.081279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] RBAC: Authorizing `identity:get_user(user_id=6084e84345db4ac7b7257332d4015324)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:03.082972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:03.085076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 587/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:37:02 2022] GET /identity/v3/users/6084e84345db4ac7b7257332d4015324 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:03.091564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:03.188355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:03.191516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7611-3fb1-49f6-999c-85d8467bbb41 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:03.225559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:03.226152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:03.226506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:03.226714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:03.291406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:03.292852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:03.392214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:03.396087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 572/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:37:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:37:03.402164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:03.471091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:03.511742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ee64-f9a5-438a-8ae6-1993a653c17b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:03.548364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:03.549003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:03.549284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:03.549557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] PATH_INFO: `/v3/projects/efec666a75724e2d86c3b57071f259a1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:03.550795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] RBAC: Authorizing `identity:get_project(project_id=efec666a75724e2d86c3b57071f259a1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:03.552163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95d1a-2748-4c2f-b124-c299945b6193 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:03.554346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 588/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:37:03 2022] GET /identity/v3/projects/efec666a75724e2d86c3b57071f259a1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:03.560624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:03.658728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed690c-70b4-4cb8-9b23-ff32a4f1655c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:03.692750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:03.693355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:03.693631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:03.693930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:03.694683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=efec666a75724e2d86c3b57071f259a1, user.id=6084e84345db4ac7b7257332d4015324) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:03.695061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:03.696495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84488ef-0e65-44f0-b339-9d495773f6aa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:03.702259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 573/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:37:03 2022] GET /identity/v3/role_assignments?user.id=6084e84345db4ac7b7257332d4015324&scope.project.id=efec666a75724e2d86c3b57071f259a1&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:05.678574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501f362-bdc0-45d9-9737-a7b5115e91e4 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:05.679145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501f362-bdc0-45d9-9737-a7b5115e91e4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:05.679145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501f362-bdc0-45d9-9737-a7b5115e91e4 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:05.680494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 589/1162] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:05.685779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:05.686244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:05.686340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:05.705111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:05.706348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:05.768888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:05.783582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 574/1163] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:06.044755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501f362-bdc0-45d9-9737-a7b5115e91e4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:06.147287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501f362-bdc0-45d9-9737-a7b5115e91e4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:06.175104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:06.181319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:06.183547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:06.183547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:06.183547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:06.250140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:06.251627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:06.348097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 590/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:37:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:37:06.354571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:06.451742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4dff-d50d-4067-9af9-3e1564622196 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:06.485872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:06.486500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:06.486812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:06.487086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:06.493686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:06.495169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-84476b03-1f38-4bc2-9064-bab806e360d8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:37:06.506595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:37:06.507962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:37:06.509034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 575/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:37:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:37:06.513926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:06.551641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:06.622638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84e68-8fd1-460b-b4bb-5fe5e3b4d383 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:06.659935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:06.660666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:06.660954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:06.661228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:06.661891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:06.662226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:06.663631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a3a32-504b-4b4a-afaa-f44859cf1851 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:06.671356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 591/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Aug 24 08:37:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:08.592348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf2a1-8a41-43a1-b75b-9fda800a33cd None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:08.592874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf2a1-8a41-43a1-b75b-9fda800a33cd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:08.592874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf2a1-8a41-43a1-b75b-9fda800a33cd None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:08.594281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 576/1167] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:08.602537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abfb3f-54ca-49c3-b89d-18d4491bc3fe None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:08.603018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abfb3f-54ca-49c3-b89d-18d4491bc3fe None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:08.603125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abfb3f-54ca-49c3-b89d-18d4491bc3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:08.621020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b1abfb3f-54ca-49c3-b89d-18d4491bc3fe None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:08.622214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b1abfb3f-54ca-49c3-b89d-18d4491bc3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:08.692815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 592/1168] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:08.979449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cf2a1-8a41-43a1-b75b-9fda800a33cd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:09.038110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:09.083613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cf2a1-8a41-43a1-b75b-9fda800a33cd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:09.119992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:09.120658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:09.120940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:09.121213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:09.121872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:09.123300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a26095-69f4-4f88-b6a0-1bd235ba1dc1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:09.216451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 577/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Wed Aug 24 08:37:08 2022] POST /identity/v3/domains => generated 267 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:37:11.200850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d463-4aab-4507-9cd2-20e15d91167c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:11.201417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d463-4aab-4507-9cd2-20e15d91167c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:11.201417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d463-4aab-4507-9cd2-20e15d91167c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:11.202852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 593/1170] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:11.208666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:11.209109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:11.209201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:11.227825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:11.229030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:11.295990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 578/1171] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:11.561671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d463-4aab-4507-9cd2-20e15d91167c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:11.587536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:11.663494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d463-4aab-4507-9cd2-20e15d91167c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:11.696891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:11.697521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:11.697803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:11.698077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] PATH_INFO: `/v3/domains/1f312cb132cf45fa9b9fa699bcdb529a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:11.709792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f312cb132cf45fa9b9fa699bcdb529a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:11.711298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba106d6-ac5f-4abb-94f2-9d625936a442 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:11.712923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 594/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Aug 24 08:37:11 2022] GET /identity/v3/domains/1f312cb132cf45fa9b9fa699bcdb529a => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:11.719123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:11.766745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:11.819907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50980-a0ac-455c-bdbd-c2858bb2e85a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:11.854624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:11.855273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:11.855554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:11.855861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:11.856504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:11.857943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b2647-f38b-46b7-8b74-6d9b753b8223 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:12.004278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 579/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:37:11 2022] POST /identity/v3/projects => generated 361 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:37:14.026313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79272-eda2-4ceb-869e-98a346b25f58 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:14.026946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79272-eda2-4ceb-869e-98a346b25f58 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:14.026946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79272-eda2-4ceb-869e-98a346b25f58 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:14.028248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 595/1174] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:14.034317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:14.034880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:14.034880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:14.054102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:14.055363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:14.127246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 580/1175] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:14.419851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c79272-eda2-4ceb-869e-98a346b25f58 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:14.503719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:14.519469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c79272-eda2-4ceb-869e-98a346b25f58 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:14.556551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:14.557183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:14.557461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:14.557733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] PATH_INFO: `/v3/domains/1f312cb132cf45fa9b9fa699bcdb529a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:14.558869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f312cb132cf45fa9b9fa699bcdb529a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:14.560293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6633f-577f-41c7-9aff-8308c16f053d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:14.561850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 596/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Aug 24 08:37:14 2022] GET /identity/v3/domains/1f312cb132cf45fa9b9fa699bcdb529a => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:14.567675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:14.594520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:14.672209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d89471-f02c-4e07-9094-8f08b7e9517f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:14.706082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:14.706722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:14.707353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:14.707628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:14.708794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:14.710234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668dd67-c3c0-457c-91f6-fd4b051ddca0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:14.798064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 581/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:37:14 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:37:16.752255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a3f1-ee91-4b41-bb82-ce66308ff69d None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:16.752807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a3f1-ee91-4b41-bb82-ce66308ff69d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:16.752807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a3f1-ee91-4b41-bb82-ce66308ff69d None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:16.754136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 597/1178] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:16.759546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:16.759983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:16.760084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:16.778792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:16.779926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:16.846200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 582/1179] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:17.131136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a3f1-ee91-4b41-bb82-ce66308ff69d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:17.234940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a3f1-ee91-4b41-bb82-ce66308ff69d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:17.268869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:17.269493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:17.269775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:17.270048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:17.280564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:37:17.285261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:37:17.286551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:37:17.287822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 598/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:37:17.292937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:17.307337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:17.390389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da4eb2-ce14-41c8-96d1-c9f5ff70e635 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:17.424400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:17.425130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:17.426109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:17.426385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:17.427117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:17.427420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:17.428807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:17.436977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 583/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:17.443522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:17.459574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:17.548453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa693c7f-a7f5-49d6-a05b-d70fcca82d79 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:17.583162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:17.583798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:17.584082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:17.584355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] PATH_INFO: `/v3/users/330aa41df7d8413bbfb965e51b63f893` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:17.597462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] RBAC: Authorizing `identity:get_user(user_id=330aa41df7d8413bbfb965e51b63f893)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:17.599022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:17.600668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 599/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:37:17 2022] GET /identity/v3/users/330aa41df7d8413bbfb965e51b63f893 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:17.606594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:17.681106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:17.707528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9674-7cb7-4244-becb-e5c1d588746e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:17.795944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:17.796659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:17.796840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:17.797111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:17.870896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:17.872406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:17.952383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:17.979222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 584/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:37:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:37:17.985216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:18.058679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:18.090110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0133-56fa-4dbf-95c5-3bc0180154d4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:18.125405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:18.126049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:18.126414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:18.126736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] PATH_INFO: `/v3/projects/4ca3e6a60c234e58a4363811c4bf8208` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:18.127918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] RBAC: Authorizing `identity:get_project(project_id=4ca3e6a60c234e58a4363811c4bf8208)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:18.129407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329eff8-7a77-45a2-873e-ebee6aea94d4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:18.131142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 600/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:37:17 2022] GET /identity/v3/projects/4ca3e6a60c234e58a4363811c4bf8208 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:18.137479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:18.236749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a09b-be60-469a-b34f-328301f50867 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:18.271307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:18.271899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:18.272176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:18.272442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:18.273158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=4ca3e6a60c234e58a4363811c4bf8208, user.id=330aa41df7d8413bbfb965e51b63f893) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:18.273461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:18.274909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14690dee-850a-4b0e-907c-ff37b8b450a9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:18.280109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 585/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:37:18 2022] GET /identity/v3/role_assignments?user.id=330aa41df7d8413bbfb965e51b63f893&scope.project.id=4ca3e6a60c234e58a4363811c4bf8208&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:20.221450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a68cf-3581-456e-9436-5b32424f673d None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:20.222026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a68cf-3581-456e-9436-5b32424f673d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:20.222026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a68cf-3581-456e-9436-5b32424f673d None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:20.223913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 601/1186] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:20.229097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af22d4-576d-48b5-8469-325331408842 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:20.229554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af22d4-576d-48b5-8469-325331408842 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:20.229651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af22d4-576d-48b5-8469-325331408842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:20.248131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e7af22d4-576d-48b5-8469-325331408842 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:20.249327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e7af22d4-576d-48b5-8469-325331408842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:20.278484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:20.316067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 586/1187] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:20.583523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a68cf-3581-456e-9436-5b32424f673d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:20.682258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a68cf-3581-456e-9436-5b32424f673d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:20.716266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:20.716892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:20.717171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:20.717438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:20.724629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:20.728303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:37:20.732887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:37:20.734063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:37:20.734980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 602/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:37:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:37:20.740286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af22d4-576d-48b5-8469-325331408842 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:20.843595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af22d4-576d-48b5-8469-325331408842 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:20.878739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:20.879915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:20.880201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:20.880471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:20.881139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:20.881438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:20.883058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:20.891644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 587/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:37:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:20.898100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:20.993198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea5d97-a258-46c7-baf4-8da72691e47c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:21.027125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:21.027739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:21.028023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:21.028298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] PATH_INFO: `/v3/users/330aa41df7d8413bbfb965e51b63f893` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:21.029397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] RBAC: Authorizing `identity:get_user(user_id=330aa41df7d8413bbfb965e51b63f893)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:21.030861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:21.032505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 603/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:37:20 2022] GET /identity/v3/users/330aa41df7d8413bbfb965e51b63f893 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:21.038460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:21.070416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:21.143127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f398d6-273b-4bf2-a07b-ff5f4a1b2571 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:21.181243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:21.181898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:21.182179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:21.182453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] PATH_INFO: `/v3/projects/4ca3e6a60c234e58a4363811c4bf8208` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:21.183723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] RBAC: Authorizing `identity:get_project(project_id=4ca3e6a60c234e58a4363811c4bf8208)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:21.185204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331d753-b9e4-4e7c-bea7-b64abb9f8c03 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:21.186978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 588/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:37:21 2022] GET /identity/v3/projects/4ca3e6a60c234e58a4363811c4bf8208 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:21.193624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:21.258464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:21.292175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c590d-0c4a-4f97-8490-48fd7135f42b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:21.326746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:21.327402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:21.327684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:21.327957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] PATH_INFO: `/v3/projects/4ca3e6a60c234e58a4363811c4bf8208/users/330aa41df7d8413bbfb965e51b63f893/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:21.329829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ca3e6a60c234e58a4363811c4bf8208, user_id=330aa41df7d8413bbfb965e51b63f893, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:21.331542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b97d04-640d-485c-9fe4-605bc8f3b9f5 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:21.434219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 604/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:37:21 2022] PUT /identity/v3/projects/4ca3e6a60c234e58a4363811c4bf8208/users/330aa41df7d8413bbfb965e51b63f893/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:37:23.391206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ba21-0ba2-4f2d-9fbc-06362158b388 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:23.391783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ba21-0ba2-4f2d-9fbc-06362158b388 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:23.391783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ba21-0ba2-4f2d-9fbc-06362158b388 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:23.393085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 589/1193] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:37:23.399081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:23.399510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:23.399690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:23.417370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:37:23.418569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:37:23.485359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 605/1194] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:37:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:37:23.765467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913ba21-0ba2-4f2d-9fbc-06362158b388 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:23.863676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:23.868412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913ba21-0ba2-4f2d-9fbc-06362158b388 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:23.903367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:23.904002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:23.904283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:23.904557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:23.915211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:37:23.919993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:37:23.921122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:37:23.922172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 590/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:37:23.927912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:23.983135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:24.025831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896063da-3dc4-4291-bddd-765847b7f9df None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:24.060042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:24.060649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:24.060927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:24.061197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:24.061856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:24.062153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:24.063549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:24.071650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 606/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:37:24.077543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:24.184854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08de7f4-96ef-465b-a846-7e99e6757f21 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:24.218309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:24.218968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:24.219250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:24.219525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] PATH_INFO: `/v3/users/330aa41df7d8413bbfb965e51b63f893` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:24.220667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=330aa41df7d8413bbfb965e51b63f893)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:24.222110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:24.224249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 591/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Aug 24 08:37:24 2022] GET /identity/v3/users/330aa41df7d8413bbfb965e51b63f893 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:24.230202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:24.334189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05cddb-128d-41eb-9a1c-ead48ba8fa96 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:24.351668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:24.369901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:24.370517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:24.370836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:24.371116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:24.440397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:24.441851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:24.538887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 607/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:37:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:37:24.544656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:24.573565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:24.651364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5693-6b21-4a3b-b82c-3fe40962c0c5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:24.685087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:24.685747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:24.686026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:24.686297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] PATH_INFO: `/v3/projects/4ca3e6a60c234e58a4363811c4bf8208` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:24.687452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] RBAC: Authorizing `identity:get_project(project_id=4ca3e6a60c234e58a4363811c4bf8208)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:24.688958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da24f-07b3-462f-8dfe-35a381eb17d1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:24.690624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 592/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Aug 24 08:37:24 2022] GET /identity/v3/projects/4ca3e6a60c234e58a4363811c4bf8208 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:24.697321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:37:24.716982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:24.797987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cd98c-b352-4e08-8816-3e52b39b9db6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:37:24.835302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:37:24.835909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:37:24.836192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:37:24.836467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:37:24.837193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=4ca3e6a60c234e58a4363811c4bf8208, user.id=330aa41df7d8413bbfb965e51b63f893) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:37:24.837503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:37:24.838975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859dd84-5977-41c0-8747-f9b54b09a914 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:37:24.844887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 608/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:37:24 2022] GET /identity/v3/role_assignments?user.id=330aa41df7d8413bbfb965e51b63f893&scope.project.id=4ca3e6a60c234e58a4363811c4bf8208&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:37:34.680633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=53 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:37:34.842256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=59 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:52.891571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c436a-ba5f-48e8-8660-f7488cacdbb5 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:52.892146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c436a-ba5f-48e8-8660-f7488cacdbb5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:52.892146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c436a-ba5f-48e8-8660-f7488cacdbb5 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:52.893489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 593/1201] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:52.899498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:52.900046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:52.900046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:52.920051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:52.921284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:52.993846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 609/1202] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:53.262536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c436a-ba5f-48e8-8660-f7488cacdbb5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:53.364272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c436a-ba5f-48e8-8660-f7488cacdbb5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:53.396990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:53.397649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:53.397930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:53.398201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:53.408751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:53.410209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:53.424997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:53.426208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:53.427600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:53.427600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:53.427600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:38:53.427600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 594/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:53.432128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:53.530617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63714-618e-404f-bce2-99bd9dcb826b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:53.558630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:53.564465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:53.566677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:53.566677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:53.566677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:53.566677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:53.566677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:53.567892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aecfb2-357a-4879-944a-afa50d5b6b26 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:53.579457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 610/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:53.585321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:53.639208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:53.684981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e6a8a-0e97-4fb3-acec-c5f0d2e440e5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:53.718946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:53.719541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:53.719818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:53.720090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:53.720727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:53.722166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88438a-81fb-4b72-938b-5e9cc51a550f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:53.837026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 595/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Aug 24 08:38:53 2022] POST /identity/v3/users => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:38:55.780901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a9e-c7e6-48ce-848d-5cea7dbd3f3a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:55.781460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a9e-c7e6-48ce-848d-5cea7dbd3f3a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:55.781460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a9e-c7e6-48ce-848d-5cea7dbd3f3a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:55.782787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 611/1206] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:55.787996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:55.788389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:55.788576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:55.807007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:55.808177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:55.874536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 596/1207] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:56.145529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a0a9e-c7e6-48ce-848d-5cea7dbd3f3a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:56.252260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a0a9e-c7e6-48ce-848d-5cea7dbd3f3a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:56.284993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:56.285617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:56.285902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:56.286179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:56.297619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:56.302642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:56.303702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:38:56.304597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 612/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:56.310071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:56.349127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:56.408206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f40bcc-fe95-4a3a-b75f-8902e50d0155 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:56.441832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:56.442512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:56.442825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:56.443101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:56.443802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:56.444103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:56.445479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:56.454259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 597/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:56.460128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:56.508263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:56.557841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e340-e98f-40e9-b107-d1af763acacc None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:56.591456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:56.592080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:56.592364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:56.592638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:56.602949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:56.604428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:56.618617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:56.619784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:56.621039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:56.621039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:56.621039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:38:56.621039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 613/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:56.626498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:56.722387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:56.725235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed7df0-d168-4a6d-ab8e-44f38ed0e3ee None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:56.758641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:56.768984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:56.772957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 598/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:56.778700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:56.881538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b6bb1-143d-459c-b4b6-3cab4738c47d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:56.883103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:56.914742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:56.915360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:56.915644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:56.915914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:56.923762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:56.925221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:56.935895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:56.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:38:56.938209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 614/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:56.943080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.042935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85fdde-e067-4dc5-935a-4376a2965b17 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:57.077040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:57.077410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:57.077728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:57.077961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:57.078710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:57.081579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:57.081579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:57.097811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 599/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:57.103974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.204272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1899-84ec-4555-999b-8b8eb45a90bd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:57.236777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:57.237427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:57.237708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:57.237980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:57.266490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:57.301716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:57.303131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:57.400761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 615/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:38:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:38:57.407376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.427385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:57.506413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c67-080c-4eaf-b16a-89702f879edc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:57.542386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:57.543115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:57.543390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:57.543660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:57.554346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:57.555849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:57.570440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:57.575674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:57.576700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:57.576700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:57.576700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:38:57.576700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 600/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:57.578496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.638428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:57.675704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1e8f3-878d-4ce0-9b69-bc8d455b4d27 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:57.709273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:57.709950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:57.710235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:57.710512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:57.711209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:57.711512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:57.712940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:57.724254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 616/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:57.730171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.799249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:57.831535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834b0e8-e85d-40cf-8c28-dde6fa395a2b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:57.864389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:57.865018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:57.865301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:57.865578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:57.873038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:57.874477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:57.886450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:57.890643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:38:57.891696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 601/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:57.894177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:57.992569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c258f3-4342-4495-9fc8-7ed20f861623 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:58.015103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:58.025919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:58.026526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:58.026853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:58.027133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:58.027820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:58.028158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:58.029612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f79d36-b1ad-42bc-8015-8c62fe2e3ac2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:58.041079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 617/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:58.047336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:58.147304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bea2b-be8e-4c37-8b11-de9e2e6a97ab None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:58.175353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:58.191006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:58.191006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:58.191006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:58.191006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:58.191415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=440669b078bd474d955d2faea0dbe859) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:58.191415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:58.191415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d48d9-353e-41a2-a463-da2acb69a494 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:58.191415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 602/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:38:58 2022] GET /identity/v3/role_assignments?user.id=440669b078bd474d955d2faea0dbe859&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:00.095465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea99e3-40e5-4a91-a7e4-9a828e142e50 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:00.096048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea99e3-40e5-4a91-a7e4-9a828e142e50 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:00.096048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea99e3-40e5-4a91-a7e4-9a828e142e50 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:00.097481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 618/1220] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:00.102617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:00.103077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:00.103208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:00.122807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:00.124004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:00.192523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 603/1221] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:00.475930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea99e3-40e5-4a91-a7e4-9a828e142e50 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:00.580069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea99e3-40e5-4a91-a7e4-9a828e142e50 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:00.613805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:00.614430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:00.614742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:00.615017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:00.625555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:39:00.630491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:00.631712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:00.632653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 619/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:00.638308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:00.733736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89545a36-c65c-43dc-be72-aefdb30f1bbd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:00.756460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:00.767349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:00.767937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:00.768216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:00.768490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:00.769153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:00.769454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:00.770863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:00.778873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 604/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:00.810066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:00.828190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:00.881549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3ac33-6935-4e79-9dbd-cb90c25c2826 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:00.914474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:00.915130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:00.922708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:00.922708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:00.925460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:00.926937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:00.941296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:00.942301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:00.943267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:00.943267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:00.943267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:00.943267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 620/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:00.948453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.047135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457c3ea-ef91-4558-b5c1-3156d7a0dc95 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.079569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.080163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.080439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.080708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.081342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:01.081638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.083016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.094815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 605/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:01.100564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.194631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:01.201379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e68852-a215-41c2-822c-616eae1c684c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.233416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.234037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.234320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.234596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.242235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.245872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:39:01.255554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:01.256696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:01.258271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 621/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:01.262561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.308506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:01.360751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7548f-ab40-4529-815d-bbea02cf874e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.398810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.399419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.399699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.399971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.400705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:01.401011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.402449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.420798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 606/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:39:01.426622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.523687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a973-fefd-47f1-b745-b0150d4139cb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.557323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.557939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.558217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.558488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.568610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.569989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:01.583538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:01.585037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:01.586441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:01.586441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:01.586441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:01.586441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 622/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:01.591228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.690936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f2a55-1ae7-443f-ab94-95d0c16ba1c3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.693310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:01.724834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.725440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.725717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.725985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.726629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:01.726960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.728327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.739841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 607/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:01.745640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:01.756110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:01.845916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23b01-b269-4978-ab46-4caf0b688273 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:01.878720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:01.879348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:01.879629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:01.879896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:01.886339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:01.888384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:39:01.899326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:01.900390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:01.901318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 623/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:01.906965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:02.006944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6363e-59e7-48ae-bbdf-f376cac15e28 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:02.041618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:02.042222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:02.042502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:02.042806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:02.043521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:02.043908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:02.045341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71c64-e4bb-4b17-b818-fd3e34886c80 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:02.057051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 608/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:02.063309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:02.127115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:02.160863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58a54f-def8-4448-bbf1-c9733fe247fb None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:02.193107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:02.193762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:02.194042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:02.194315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/440669b078bd474d955d2faea0dbe859/roles/d14f58edfef640269b655072f73f88b3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:02.208723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=440669b078bd474d955d2faea0dbe859, role_id=d14f58edfef640269b655072f73f88b3)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:02.210150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a112069-b386-4a1c-8f39-def66628426c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:02.444390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 624/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:39:02 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/440669b078bd474d955d2faea0dbe859/roles/d14f58edfef640269b655072f73f88b3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:39:04.307415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34f705-9af4-4cf6-b902-6a2e1fa3e984 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:04.307997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34f705-9af4-4cf6-b902-6a2e1fa3e984 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:04.307997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34f705-9af4-4cf6-b902-6a2e1fa3e984 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:04.309928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 609/1233] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:04.315166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:04.315553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:04.315723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:04.334140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:04.335337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:04.401775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 625/1234] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:04.678747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34f705-9af4-4cf6-b902-6a2e1fa3e984 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:04.703941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:04.778815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34f705-9af4-4cf6-b902-6a2e1fa3e984 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:04.811750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:04.812373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:04.812654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:04.812925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:04.823045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:39:04.828388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:04.829524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:04.830640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 610/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Aug 24 08:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:04.835565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:04.879652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:04.930902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d116a-b5cb-40eb-82bd-6b125312f00c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:04.964377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:04.965629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:04.965911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:04.966187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:04.966876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:04.967179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:04.968585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:04.976171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 626/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Aug 24 08:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:04.982148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.067690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:05.083731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af540e6-577f-4091-809f-94cac6394527 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:05.117222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:05.117848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:05.118130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:05.118403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:05.129016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:05.130570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:05.146071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:05.147412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:05.148584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:05.148584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:05.148584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:05.148584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 611/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:05.153460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.255440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c9f8-0bba-42a2-83bd-96d2a0db58b2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:05.264873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:05.288460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:05.289061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:05.289341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:05.289617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:05.290259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:05.290554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:05.291933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:05.304110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 627/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:05.310642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.411129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb71b1-6a0d-4a3b-ade1-8907d590fa5e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:05.444532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:05.445160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:05.445434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:05.445702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:05.448422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:05.454335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:05.455862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:39:05.466793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:05.478847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:05.479872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 612/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:39:05.480508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.575005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b2cb9-12ef-4609-a6c6-3e9eb4281097 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:05.607309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:05.607907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:05.608180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:05.608448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:05.609162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:05.609457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:05.610892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:05.627562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 628/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:39:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:05.633559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.734632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fac-dbbb-4a52-ba84-a6dad0790cab None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:05.768428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:05.769055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:05.769332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:05.769602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:05.830244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:05.836463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:05.837916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:05.935440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 613/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:39:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:39:05.941727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:05.971609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:06.041261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f941a24-86c3-4e31-8439-1972391baad3 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:06.074010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:06.074941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:06.075224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:06.075493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:06.086283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:06.086508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:06.101076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:06.102262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:06.103436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:06.103436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:06.103436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:06.103436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 629/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:39:06.108694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:06.206217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:06.212085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6dd8c-b85d-4f9a-89c8-372e41ada260 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:06.245476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:06.246146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:06.246677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:06.246963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:06.247608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:06.247912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:06.262976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:06.262976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 614/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:06.267080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:06.345196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:06.365545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bd72-cf74-4862-a5c0-254aef8e45f7 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:06.398077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:06.398724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:06.399007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:06.399276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:06.406414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:06.407878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:39:06.418742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:06.419645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:06.420767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 630/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:06.426085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:06.528909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214985-5a2d-4bb1-a3e3-13a2a81be8e8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:06.562744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:06.563351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:06.563633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:06.563906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:06.564609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:06.564913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:06.566346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728999fd-0863-456f-a75d-5e5c2cb8a35e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:06.579339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 615/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:06.585520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:06.685836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884cc9f-051d-4806-9644-7a0ce25cc1ce None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:06.719290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:06.719892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:06.720175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:06.720449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:06.721172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] RBAC: Adding query filter params (role.id=d14f58edfef640269b655072f73f88b3, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=440669b078bd474d955d2faea0dbe859) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:06.721479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:06.722964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b9f9-0ef7-4851-9f23-5995cf7f4d3e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:06.729200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 631/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:39:06 2022] GET /identity/v3/role_assignments?user.id=440669b078bd474d955d2faea0dbe859&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=d14f58edfef640269b655072f73f88b3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:08.687792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773bdf-a199-4dda-91d1-286e652dd958 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:08.688349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773bdf-a199-4dda-91d1-286e652dd958 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:08.688349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e773bdf-a199-4dda-91d1-286e652dd958 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:08.689686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 616/1247] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:08.694973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:08.695390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:08.695531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:08.701953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:08.714305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:08.715908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:08.780831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 632/1248] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:09.057210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773bdf-a199-4dda-91d1-286e652dd958 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.076098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:09.159136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e773bdf-a199-4dda-91d1-286e652dd958 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.193293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.193921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.194199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.194473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.206243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:39:09.214169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:09.215391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:09.216836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 617/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:09.221869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.316668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15ad37-9ad4-495d-af00-a0e2e833b120 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.344780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:09.349428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.350032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.351665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.351665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.351665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:09.351665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:09.352844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:09.360547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 633/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:09.366351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.443479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:09.466468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72ea6c-6fd7-4635-93dd-456f7cd8a38f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.500176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.500810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.501090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.501363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.513236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:09.514826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:09.529263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:09.537593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:09.538627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:09.538627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:09.538627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:09.538627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 618/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:09.538896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.633916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122802f-90ad-4576-aacc-eafff933ac35 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.665063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.665653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.665933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.666202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.666858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:09.667154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:09.669033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:09.680109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 634/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:09.686369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.784435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209765-db27-4edb-ad53-11efb4b00363 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.810646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:09.818005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.818627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.818926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.819194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.826733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:09.828157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:39:09.839086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:09.842258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:09.843306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 619/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:39:09.846219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:09.901654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:09.944467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057774-646b-4c5c-ba64-5c9391fc21b4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:09.977570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:09.978177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:09.978462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:09.978913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:09.979559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:09.979866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:09.981970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:09.998401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 635/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:39:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:10.004324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.103077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f796f-bea2-4c78-8dd5-c12de1fbeb82 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:10.136396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:10.137000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:10.137273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:10.137618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:10.192837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:10.205635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:10.207213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:10.303519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 620/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:39:10.309426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.407420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6215c-e7ef-4e0b-b675-2f8b37c4b74e None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:10.420921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:10.442712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:10.443398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:10.443635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:10.443910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:10.454006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:10.455494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:10.470363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:10.471618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:10.472563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:10.472563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:10.472563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:10.472563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 636/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:10.477446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.570029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:10.581873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e261c2-f763-439a-8bca-5f3c421c594c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:10.616321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:10.617039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:10.617325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:10.617601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:10.618242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:10.618549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:10.619985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:10.631558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 621/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:10.637422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.737154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546297f7-3dca-4f97-b6a3-773bbd37be4f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:10.772092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:10.772727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:10.773009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:10.773282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:10.780825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:10.782279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:39:10.794946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:10.796137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:10.797108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 637/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:10.802192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.905106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51f72-0bdc-4415-938a-f1a331b7b3d1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:10.937525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:10.938121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:10.938395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:10.938692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:10.943623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:10.943623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:10.943623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60987a4e-df32-42eb-8afc-48662b00fe62 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:10.952985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 622/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:10.959409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:10.989325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:11.061307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2fae-94b1-434e-927e-b7564ce8885c None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:11.094548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:11.095171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:11.095449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:11.095719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:11.096437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=440669b078bd474d955d2faea0dbe859) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:11.096740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:11.098182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c1e0-8859-403a-8452-0ba779d33824 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:11.103403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 638/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:39:10 2022] GET /identity/v3/role_assignments?user.id=440669b078bd474d955d2faea0dbe859&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:12.999220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673b58-7adf-4107-b9db-5696f0647d64 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:12.999828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673b58-7adf-4107-b9db-5696f0647d64 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:12.999828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74673b58-7adf-4107-b9db-5696f0647d64 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.001192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 623/1261] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:13.006374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:13.006819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:13.006969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.025654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:13.033527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:13.091638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 639/1262] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:13.364519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74673b58-7adf-4107-b9db-5696f0647d64 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:13.375735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:13.464961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74673b58-7adf-4107-b9db-5696f0647d64 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:13.498620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:13.499269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:13.499552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:13.499850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.509733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:39:13.515064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:13.516308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:13.517427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 624/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:13.522470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:13.541596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:13.620465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bdd77-ffdc-40b8-9a6b-0d5c2303555f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:13.655287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:13.655894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:13.656177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:13.656451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.657113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:13.657414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:13.658832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:13.666548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 640/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:13.672475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:13.738015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:13.770052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d55b7-5903-41ad-a96a-eb48efa391fc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:13.801446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:13.802060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:13.802335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:13.802604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.813197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:13.814632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:13.829825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:13.831035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:13.832141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:13.832141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:13.832141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:13.832141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 625/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:13.837157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:13.916073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:13.933842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7406b6-6a4d-433d-8331-644927b2889f None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:13.966891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:13.967487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:13.967770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:13.968044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:13.968681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:13.968980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:13.970347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:13.982923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 641/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:13.988778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.090336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45ead-968d-45a3-9df5-94e0a4b1115b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.110382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:14.124644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.125403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.125680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.125947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.133645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.135139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:39:14.146329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:14.147559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:14.148955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 626/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:14.153475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.252521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70fdb-fc79-437a-9e4c-fe7476afbbfa None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.287100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.292095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.299787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:14.308027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 642/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:39:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:14.314133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.412876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0062834-4349-47a8-b78f-a2711be16863 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.445130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.445753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.446029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.446297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.457282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.458785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:14.473530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:14.474501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:14.475562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:14.475562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:14.475562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:14.475562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 627/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:14.480734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.580732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d103c0-3580-449a-ac44-80e6ccf88717 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.619956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.620565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.620853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.621127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.622297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:14.622598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.624022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.634958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 643/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:14.641234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.673645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:14.739389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd73cc-3498-4f45-abe1-5091bb71b289 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.773306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.773881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.774189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.774460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.782056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.783678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:39:14.795641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:14.796755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:14.797748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 628/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:14.802482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:14.859502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:14.900766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74ac3-b332-4a55-a814-d6b09a688e0d None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:14.933609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:14.934242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:14.934527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:14.934833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:14.935538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:14.935847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:14.937272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400616-860b-4a16-b02f-11b162e4a3ab None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:14.950134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 644/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:14.956123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:15.045082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:15.053698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018c65-eda1-4dff-bdbb-14eb4eb99221 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:15.087763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:15.088429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:15.088701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:15.088978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] PATH_INFO: `/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/440669b078bd474d955d2faea0dbe859/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:15.091671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8af04616519544be965cfabfe2f8dbbb, user_id=440669b078bd474d955d2faea0dbe859, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:15.096439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efed78-1170-45ad-b2bc-f5ec085c58d9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:15.213996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 629/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Aug 24 08:39:14 2022] PUT /identity/v3/projects/8af04616519544be965cfabfe2f8dbbb/users/440669b078bd474d955d2faea0dbe859/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:39:17.137018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a1b2-29ca-438a-ac9c-63535052faea None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:17.137686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a1b2-29ca-438a-ac9c-63535052faea None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:17.137686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a1b2-29ca-438a-ac9c-63535052faea None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:17.138981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 645/1274] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:17.144786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:17.154312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:17.154312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:17.163465 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:17.164666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:17.237312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 630/1275] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:17.511174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65a1b2-29ca-438a-ac9c-63535052faea None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:17.614155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65a1b2-29ca-438a-ac9c-63535052faea None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:17.631299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:17.648546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:17.649201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:17.649487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:17.649764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:17.660536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:39:17.665203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:17.666544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:17.667842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 646/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Aug 24 08:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:17.673031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:17.750946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:17.768186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c2c9b-aec3-46b5-aa32-7441d52a83a1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:17.801085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:17.801674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:17.801950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:17.802219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:17.802898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:17.803192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:17.804587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:17.812416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 631/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Aug 24 08:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:17.818754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:17.919588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3ec27-dea0-46c2-ba40-b60abed370a9 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:17.952352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:17.952985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:17.953274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:17.953544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:17.963261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:17.965807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:17.979166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:17.980418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:17.981571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:17.981571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:17.981571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:17.981571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 647/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:17.986283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:18.085772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef513-9922-4828-ba8f-2e21a9f1d3be None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:18.118610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:18.119229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:18.119502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:18.119769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:18.120404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:18.120695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:18.122090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:18.134324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 632/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:18.140948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:18.171612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:18.238522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96181f5-7fa9-49c2-88eb-67bb4a85b33b None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:18.271929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:18.272552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:18.272836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:18.273113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] PATH_INFO: `/v3/users/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:18.280988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:18.282418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:39:18.293792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:18.295019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:18.296266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 648/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:18.300888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:18.304671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:18.397369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e38e9-eb05-4faf-9a56-c508252da251 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:18.431278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:18.431874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:18.432154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:18.432429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:18.433204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:18.433721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:18.435466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:18.453192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 633/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Aug 24 08:39:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:18.459298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:18.550612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:18.559801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f7c9b-dfca-4065-972b-c367ff7debdd None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:18.593281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:18.593883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:18.594175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:18.594438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:18.659448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:18.660904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:18.746224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:18.757704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 649/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Aug 24 08:39:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:39:18.762596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:18.822306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:18.861505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207689f5-f4a8-4eed-92de-d0c249c63133 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:18.894341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:18.894995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:18.895283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:18.895552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:18.905860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:18.907338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:39:18.924881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:18.926063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:39:18.927242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:39:18.927242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:39:18.927242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:18.927242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 634/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:18.929621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:19.028766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d7d3-37b3-43da-8d22-d431a3c72d94 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:19.062896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:19.063503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:19.063787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:19.064062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:19.064700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:19.065001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:19.066368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:19.078549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 650/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Aug 24 08:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:39:19.084353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:19.185306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b26ae2-78a6-4542-b789-2aac04e3b83e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:19.200454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:19.220064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:19.220686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:19.220965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:19.221238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:19.227896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:19.229348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:39:19.241166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:19.242292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Aug 24 08:39:19.243551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 635/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Aug 24 08:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:19.248654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:19.298389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:19.347195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1e3e4-a706-44e1-bddf-396f510a57d1 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:19.379975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:19.380575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:19.380860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:19.381181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:19.381908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:19.382182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:19.383645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba3d5-1c3d-422e-a180-0b76a4a635f8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:19.395148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 651/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Aug 24 08:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:19.401417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:19.503571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eb6da-e881-4169-927c-01b12a5fae6d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:19.536404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:19.537015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:19.537296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:19.537645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:19.538379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] RBAC: Adding query filter params (role.id=3f587a5233884fd6861867344e2c084b, scope.project.id=8af04616519544be965cfabfe2f8dbbb, user.id=440669b078bd474d955d2faea0dbe859) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:19.538714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:19.540167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf16da-fa74-4bd1-b274-89405cc0528d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:19.546510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 636/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Aug 24 08:39:19 2022] GET /identity/v3/role_assignments?user.id=440669b078bd474d955d2faea0dbe859&scope.project.id=8af04616519544be965cfabfe2f8dbbb&role.id=3f587a5233884fd6861867344e2c084b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:21.488400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6847f4-2bb3-4823-be70-777c7f2a0338 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:21.488974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6847f4-2bb3-4823-be70-777c7f2a0338 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:21.488974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6847f4-2bb3-4823-be70-777c7f2a0338 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:21.490327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 652/1288] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:21.495710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:21.496137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:21.496389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:21.514219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:21.515441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:21.545839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:21.582916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 637/1289] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:21.857983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6847f4-2bb3-4823-be70-777c7f2a0338 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:21.957686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6847f4-2bb3-4823-be70-777c7f2a0338 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:21.969786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:21.992724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:21.993353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:21.993634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:21.993907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:21.994484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:21.996388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:22.004831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:22.009942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:22.012482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 653/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:39:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:22.014337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:22.112193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20ac88-6f39-42c5-9485-a0b30aa17698 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:22.148144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f382-4228-48df-8990-a166885927f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:22.148811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f382-4228-48df-8990-a166885927f8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:22.149100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f382-4228-48df-8990-a166885927f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:22.149375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f382-4228-48df-8990-a166885927f8 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:22.150013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692f382-4228-48df-8990-a166885927f8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:22.150318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692f382-4228-48df-8990-a166885927f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:22.151762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692f382-4228-48df-8990-a166885927f8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:22.156700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 638/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Aug 24 08:39:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:22.162765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:22.262811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056259-6460-4cca-8aa4-bbd104db93a2 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:22.296260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:22.296953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:22.297240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:22.297697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] PATH_INFO: `/v3/services` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:22.298330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] RBAC: Adding query filter params (type=placement) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:22.298632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:22.307747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7b73-76b0-41d3-ba7b-ad7326d7ca1b None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:22.307747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 654/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Aug 24 08:39:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:24.221028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084fce7-a61e-4549-8314-98adadd69e0b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:24.221592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084fce7-a61e-4549-8314-98adadd69e0b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:24.221592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084fce7-a61e-4549-8314-98adadd69e0b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:24.222953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 639/1293] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:24.228244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933cd81-34f0-4c9c-a0f4-64f0fff0515a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:24.228689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933cd81-34f0-4c9c-a0f4-64f0fff0515a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:24.228783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933cd81-34f0-4c9c-a0f4-64f0fff0515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:24.247405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b933cd81-34f0-4c9c-a0f4-64f0fff0515a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:24.248593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b933cd81-34f0-4c9c-a0f4-64f0fff0515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:24.279004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:24.315230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 655/1294] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:24.583834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084fce7-a61e-4549-8314-98adadd69e0b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:24.615150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:24.689175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084fce7-a61e-4549-8314-98adadd69e0b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:24.723678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:24.724539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:24.725047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:24.725345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:24.726634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:24.728057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f7d1-a4d3-4c19-b056-36ac879ab652 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:24.834160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 640/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Aug 24 08:39:24 2022] POST /identity/v3/services => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:39:26.760798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491561d6-f28a-4627-83df-ded66befbd45 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:26.761367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491561d6-f28a-4627-83df-ded66befbd45 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:26.761367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491561d6-f28a-4627-83df-ded66befbd45 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:26.762911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 656/1296] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:26.768412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:26.768871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:26.768980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:26.787258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:26.788377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:26.860609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 641/1297] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:27.126416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491561d6-f28a-4627-83df-ded66befbd45 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:27.227664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491561d6-f28a-4627-83df-ded66befbd45 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:27.261908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:27.262591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:27.262928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:27.263206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:27.263847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:27.265210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:27.274241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:27.275530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:27.277386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 657/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:39:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:27.282165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:27.300600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:27.378532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f7915-79f2-4e7d-aef2-d21c7c0303e0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:27.411499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:27.412203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:27.412482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:27.412752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:27.413379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:27.413675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:27.415078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:27.420606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 642/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Aug 24 08:39:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:27.426544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:27.460367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:27.527052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df926bd-cb42-4722-a5b0-5c4aebfa60f6 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:27.561551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:27.562160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:27.562442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:27.562747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:27.563486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] RBAC: Adding query filter params (interface=public, service_id=65645d385ac1495389ac2479d411717e, region_id=RegionOne) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:27.563811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:27.565234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ebfdd-5445-406e-968e-ea249c8853e8 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:27.570190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 658/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Aug 24 08:39:27 2022] GET /identity/v3/endpoints?service_id=65645d385ac1495389ac2479d411717e&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:27.577254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:27.658444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:27.677275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aeea8-73d4-47fd-924b-0a26a7096db7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:27.710767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:27.711378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:27.711676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:27.711931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:27.712537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:27.713933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649e62f-a7a3-4218-93b2-addd96379df8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:27.720683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 643/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Aug 24 08:39:27 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:39:29.609130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9944e-7660-41a3-81ac-3bc0e14d6808 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:29.609721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9944e-7660-41a3-81ac-3bc0e14d6808 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:29.609834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9944e-7660-41a3-81ac-3bc0e14d6808 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:29.611256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 659/1302] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:39:29.616606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:29.617057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:29.617165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:29.634884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:39:29.636104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:39:29.703088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 644/1303] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:39:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:39:29.979954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9944e-7660-41a3-81ac-3bc0e14d6808 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:30.085468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:30.093279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9944e-7660-41a3-81ac-3bc0e14d6808 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:30.129858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:30.130510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:30.130807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:30.131188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:30.132084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:30.133799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:39:30.144722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:39:30.146312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application with Lock( Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: ERROR keystone.server.flask.application Aug 24 08:39:30.147511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 660/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Aug 24 08:39:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:39:30.152845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:30.237658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:30.263469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2922d75-9a1f-4fa3-96b9-2338e9ed1d57 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:30.301386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:30.302004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:30.302291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:30.302603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:30.303271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:39:30.303609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:30.304993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769231d-646a-4fd4-974b-be311eb6dc89 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:30.310960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 645/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Aug 24 08:39:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:39:30.316859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:39:30.422219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749edc81-e475-4cc8-84bb-771bde6694f4 None admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:39:30.457273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:39:30.457922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:39:30.458205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:39:30.458480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:39:30.459184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:39:30.460593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff9a2c-cfe2-47c5-b8c1-b35b9537de2e None admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:39:30.651324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 661/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:39:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:39:40.307157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=30 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:39:40.646391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:45.436646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ee41-8041-4279-bdc7-5c8b6848b5c2 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:45.437229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ee41-8041-4279-bdc7-5c8b6848b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:45.437629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ee41-8041-4279-bdc7-5c8b6848b5c2 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:45.439977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 646/1307] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:47:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:45.449303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c1a5-87c7-40ca-a9a1-8805816dff08 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:45.449807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c1a5-87c7-40ca-a9a1-8805816dff08 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:45.450244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c1a5-87c7-40ca-a9a1-8805816dff08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:45.604492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2ff2c1a5-87c7-40ca-a9a1-8805816dff08 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:45.605863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2ff2c1a5-87c7-40ca-a9a1-8805816dff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:45.742404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 662/1308] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:47:46.046305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.046869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.047252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.048703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 647/1309] 10.222.0.64 () {54 vars in 965 bytes} [Wed Aug 24 08:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:46.055237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0381-dbaa-4c27-82b3-11c7333e90ae None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.055613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0381-dbaa-4c27-82b3-11c7333e90ae None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.055957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0381-dbaa-4c27-82b3-11c7333e90ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.139304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c67b0381-dbaa-4c27-82b3-11c7333e90ae None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:46.140577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c67b0381-dbaa-4c27-82b3-11c7333e90ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:46.168837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:46.240750 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 663/1310] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:46.250540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:46.252368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:46.385492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade301-56cf-498c-925d-2bd9c81ec1a1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:46.469181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:46.470050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.470050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.470457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.471989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:46.542623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:46.612794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:46.616264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95881c28-33de-4959-bc8c-a24057c22c80 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:46.642479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 648/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:49.273615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb132-5cee-4448-a6d0-4bb6441c5bd1 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:49.274237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb132-5cee-4448-a6d0-4bb6441c5bd1 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:49.274237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb132-5cee-4448-a6d0-4bb6441c5bd1 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:49.276328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 664/1312] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:49.282379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feddf4-4694-4a84-903c-379ebdfea0b5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:49.283048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feddf4-4694-4a84-903c-379ebdfea0b5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:49.283048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feddf4-4694-4a84-903c-379ebdfea0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:49.370262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-22feddf4-4694-4a84-903c-379ebdfea0b5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:49.371556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-22feddf4-4694-4a84-903c-379ebdfea0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:49.479353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 649/1313] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:47:49.786628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:49.786938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:49.787216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:49.788597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 665/1314] 10.222.0.64 () {54 vars in 965 bytes} [Wed Aug 24 08:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:49.794979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac6643-e5b4-4477-b0dd-de4dfd73ae7a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:49.795267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac6643-e5b4-4477-b0dd-de4dfd73ae7a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:49.795547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac6643-e5b4-4477-b0dd-de4dfd73ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:49.807862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:49.880739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-eeac6643-e5b4-4477-b0dd-de4dfd73ae7a None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:49.881943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eeac6643-e5b4-4477-b0dd-de4dfd73ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:49.964280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 650/1315] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:49.973733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:49.975344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:50.101315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cf837-fc90-478c-9189-47d0514f4e49 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:50.176391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:50.179017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:50.179641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:50.179925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:50.180202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:50.181842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:50.296187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:50.297646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195fb82-8d95-4f7b-b195-17ccef39fe05 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:50.317955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 666/1316] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:53.152389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:53.152936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:53.152936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:53.154229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 651/1317] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:53.160263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd17a7-f4bb-4ccb-a71b-cca4a26822de None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:53.160749 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd17a7-f4bb-4ccb-a71b-cca4a26822de None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:53.160846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd17a7-f4bb-4ccb-a71b-cca4a26822de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:53.229041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4fbd17a7-f4bb-4ccb-a71b-cca4a26822de None None] MFA Rules not processed for user `5b57159c4512494f809abfa825aef1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:53.230258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4fbd17a7-f4bb-4ccb-a71b-cca4a26822de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:53.291302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:53.337640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 667/1318] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:53.635986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:53.637597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:53.741575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420fa92-dc27-4253-ad7e-9f417eb7341f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:53.744475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:53.818694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:53.819245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:53.819545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:53.819799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:53.821554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:53.935122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:53.936581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d66af6-463d-46a8-80ee-6059e6ecdd75 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:53.954869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 652/1319] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:56.664989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:56.665582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:56.665582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:56.667141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 668/1320] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:47:56.672926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086290-5a06-4541-b7f4-1d2f17d194f9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:56.673417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086290-5a06-4541-b7f4-1d2f17d194f9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:56.673553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb086290-5a06-4541-b7f4-1d2f17d194f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:56.721969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:56.751683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cb086290-5a06-4541-b7f4-1d2f17d194f9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:56.754071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cb086290-5a06-4541-b7f4-1d2f17d194f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:56.860552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 653/1321] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:57.360621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:57.362142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:57.479561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052d259-2048-4295-845a-85a4a77b3474 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:57.592943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:57.593615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:57.593939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:57.594254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:57.596015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:57.605135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:57.720066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:57.721547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3cdc0-f6a8-439a-ba72-b8f2e10a8111 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:57.750859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 669/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:01.250038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:01.250681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:01.250681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:01.252217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 654/1323] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:01.261165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194f9cb-577e-4b08-b9d5-716e30dc385c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:01.261745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194f9cb-577e-4b08-b9d5-716e30dc385c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:01.262150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194f9cb-577e-4b08-b9d5-716e30dc385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:01.363464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f194f9cb-577e-4b08-b9d5-716e30dc385c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:01.364747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f194f9cb-577e-4b08-b9d5-716e30dc385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:01.477674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:01.554044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 670/1324] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:01.844375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:01.845989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:01.960328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b77f7-b4a7-4e86-afb3-ccaf9a7ffade None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:01.973844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:02.045897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:02.046417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:02.046725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:02.046997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:02.048639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:02.168791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:02.170290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2c9d-a661-4645-a7d5-a45af246a1dc service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:02.190718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 655/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:04.223845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:04.224396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:04.224396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:04.225763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 671/1326] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:04.231714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b7e12-c2bc-4b4c-affa-b74793805f7c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:04.232228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b7e12-c2bc-4b4c-affa-b74793805f7c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:04.232425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b7e12-c2bc-4b4c-affa-b74793805f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:04.289215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:04.313009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-221b7e12-c2bc-4b4c-affa-b74793805f7c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:04.314309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-221b7e12-c2bc-4b4c-affa-b74793805f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:04.428131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 656/1327] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:04.712908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:04.714474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:04.831987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a378f3-c935-403e-a064-3dc0949d5b8f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:04.911460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:04.912066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:04.912344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:04.912613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:04.914215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:04.935510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:05.056240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:05.058672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92c97-e806-42d8-8ba9-46e408651ea7 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:05.084133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 672/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:07.142055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:07.142859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:07.142859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:07.143888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 657/1329] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:07.150111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55749-65eb-400e-bab9-0aa05654b4f4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:07.150725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55749-65eb-400e-bab9-0aa05654b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:07.150725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f55749-65eb-400e-bab9-0aa05654b4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:07.220838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e3f55749-65eb-400e-bab9-0aa05654b4f4 None None] MFA Rules not processed for user `5b57159c4512494f809abfa825aef1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:07.222110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e3f55749-65eb-400e-bab9-0aa05654b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:07.293583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:07.335611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 673/1330] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:07.622123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:07.623831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:07.749270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393bdaf-b592-42ab-a72b-37a36bc74852 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:07.773158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:07.840400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:07.840901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:07.841182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:07.841454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:07.843190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:07.974468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:07.976117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62256ed8-1721-4c8c-9667-392b864915b0 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:07.999694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 658/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:10.692912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:10.693487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:10.693487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:10.694848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 674/1332] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:10.701256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aba71-bea2-4651-bf2a-c2d8c74f0a7d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:10.701758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aba71-bea2-4651-bf2a-c2d8c74f0a7d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:10.701861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aba71-bea2-4651-bf2a-c2d8c74f0a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:10.769118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:10.772108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f08aba71-bea2-4651-bf2a-c2d8c74f0a7d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:10.773449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f08aba71-bea2-4651-bf2a-c2d8c74f0a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:10.891023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 659/1333] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:11.182216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:11.184481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:11.308678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b513d6-39ef-4667-bbc2-cad2a00f8c9c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:11.388436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:11.389061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:11.389348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:11.389628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:11.391293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:11.422592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:11.521366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:11.524211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c112425-06a0-445a-8667-54bd67d49b0d service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:11.560588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 675/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:13.668471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:13.669072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:13.669072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:13.670409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 660/1335] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:13.676385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef5b6-ef45-4c87-8b4a-aac7cc616ce5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:13.676852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef5b6-ef45-4c87-8b4a-aac7cc616ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:13.676973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef5b6-ef45-4c87-8b4a-aac7cc616ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:13.746739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7aeef5b6-ef45-4c87-8b4a-aac7cc616ce5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:13.748095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7aeef5b6-ef45-4c87-8b4a-aac7cc616ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:13.828490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:13.852711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 676/1336] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:14.141135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:14.142758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:14.277883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacf803-23f4-4e13-8bf6-81cd87b94b47 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:14.318292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:14.367233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:14.367881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:14.368158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:14.368426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:14.370048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:14.487434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:14.488903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad638ff-7a9e-407c-978a-96b7677d5096 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:14.509702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 661/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:17.489822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:17.490471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:17.490471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:17.491836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 677/1338] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:17.498150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1189-241e-49f6-a453-6b39c684c20c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:17.498687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1189-241e-49f6-a453-6b39c684c20c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:17.498933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1189-241e-49f6-a453-6b39c684c20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:17.580816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-522d1189-241e-49f6-a453-6b39c684c20c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:17.582105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-522d1189-241e-49f6-a453-6b39c684c20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:17.592363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:17.691164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 662/1339] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:17.972905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:17.974475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:18.080646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8ed19-6693-4a86-b86a-e5e6f472749d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:18.160534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:18.161181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.161481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.161777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.163554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:18.203408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:18.285774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:18.287180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555c3e9-9aba-49e4-aef3-72d15ff56601 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:18.308592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 678/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:22.701588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.702365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.702606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.704589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 663/1341] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:22.710802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b878ef-9735-4d24-88e0-69766c3d41d3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.711590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b878ef-9735-4d24-88e0-69766c3d41d3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.711590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b878ef-9735-4d24-88e0-69766c3d41d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.800344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-43b878ef-9735-4d24-88e0-69766c3d41d3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:22.802385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-43b878ef-9735-4d24-88e0-69766c3d41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:22.919645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:22.932340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 679/1342] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:23.224146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:23.225777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:23.348547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11492400-4e58-4c72-9b2c-c60f6f29fad3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:23.391740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:23.435145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:23.435788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:23.436091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:23.436403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:23.438217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:23.575023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:23.576388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edc7a8-b99d-402e-8fd9-5962c12897db service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:23.599853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 664/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:26.989835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:26.990398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:26.990489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:26.991815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 680/1344] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:26.997358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caef13-bfa3-479b-97d6-721023a24cd4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:26.997828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caef13-bfa3-479b-97d6-721023a24cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:26.997921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caef13-bfa3-479b-97d6-721023a24cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:27.070882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d8caef13-bfa3-479b-97d6-721023a24cd4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:27.072081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d8caef13-bfa3-479b-97d6-721023a24cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:27.101408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:27.218491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 665/1345] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:27.544567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:27.546104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:27.667163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64850272-71e3-4023-94a5-1d10202f2e7c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:27.765170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:27.765908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:27.766183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:27.766477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:27.768164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:27.820665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:27.896765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:27.898233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff7a3a-0c76-493d-8f4d-6a86471b873e service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:27.918559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 681/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:32.656127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:32.656796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:32.656796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:32.658099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 666/1347] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:32.664740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766390cd-2abe-4694-a106-64539baea090 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:32.665997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766390cd-2abe-4694-a106-64539baea090 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:32.665997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766390cd-2abe-4694-a106-64539baea090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:32.743519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-766390cd-2abe-4694-a106-64539baea090 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:32.744882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-766390cd-2abe-4694-a106-64539baea090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:32.896786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:32.898803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 682/1348] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:33.191788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:33.193892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:33.328776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1dde-0499-4957-b207-b870b58b2382 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:33.386189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:33.417121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:33.417749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:33.418032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:33.418288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:33.419973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:33.556057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:33.557155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98143c-df4b-4542-9261-fd91cba694b3 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:33.577236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 667/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:39.023329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca787-618e-41a4-88f3-9b21815e10b4 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:39.023888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca787-618e-41a4-88f3-9b21815e10b4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:39.023888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca787-618e-41a4-88f3-9b21815e10b4 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:39.025990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 683/1350] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:39.032330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e26e2-e2f9-456b-b250-710c48fce77a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:39.032330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e26e2-e2f9-456b-b250-710c48fce77a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:39.032330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e26e2-e2f9-456b-b250-710c48fce77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:39.192416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d51e26e2-e2f9-456b-b250-710c48fce77a None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:39.197780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d51e26e2-e2f9-456b-b250-710c48fce77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:39.227235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:39.434971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 668/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:44.498848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507a94-dd64-4f34-af02-7641fd513006 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:44.499822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507a94-dd64-4f34-af02-7641fd513006 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:44.500092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507a94-dd64-4f34-af02-7641fd513006 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:44.501963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 684/1352] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:44.508136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252a7e-1fe7-461e-9387-53c264e5b86c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:44.508923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252a7e-1fe7-461e-9387-53c264e5b86c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:44.508923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3252a7e-1fe7-461e-9387-53c264e5b86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:44.640654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f3252a7e-1fe7-461e-9387-53c264e5b86c None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:44.642358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f3252a7e-1fe7-461e-9387-53c264e5b86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:44.734633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:44.794363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 669/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:46.209272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392928-66ac-4913-9028-8a253eeb2803 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.209992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392928-66ac-4913-9028-8a253eeb2803 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.210511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19392928-66ac-4913-9028-8a253eeb2803 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.213255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 685/1354] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:48:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:46.221519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e054-196f-46f5-b100-b8b149c88aba None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.222991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e054-196f-46f5-b100-b8b149c88aba None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.222991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e054-196f-46f5-b100-b8b149c88aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.313492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a544e054-196f-46f5-b100-b8b149c88aba None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:46.314758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a544e054-196f-46f5-b100-b8b149c88aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:46.456115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 670/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:49.920374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe6d7c-e49a-41a5-ab49-c353ea348391 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:49.920936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe6d7c-e49a-41a5-ab49-c353ea348391 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:49.920936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe6d7c-e49a-41a5-ab49-c353ea348391 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:49.922792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 686/1356] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:49.929188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf255e4-bd23-4229-8ddc-d44739ad02f8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:49.929648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf255e4-bd23-4229-8ddc-d44739ad02f8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:49.930279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf255e4-bd23-4229-8ddc-d44739ad02f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:49.969892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:50.020588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5cf255e4-bd23-4229-8ddc-d44739ad02f8 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:50.021868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5cf255e4-bd23-4229-8ddc-d44739ad02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:50.186681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 671/1357] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:50.485506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:50.485787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:50.486231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:50.487797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 687/1358] 10.222.0.64 () {54 vars in 962 bytes} [Wed Aug 24 08:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:50.496419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455ba0-07f6-4ebc-9b31-ec336600bf31 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:50.496704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455ba0-07f6-4ebc-9b31-ec336600bf31 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:50.496986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56455ba0-07f6-4ebc-9b31-ec336600bf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:50.598412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-56455ba0-07f6-4ebc-9b31-ec336600bf31 None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:50.599654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56455ba0-07f6-4ebc-9b31-ec336600bf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:50.661020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:50.734312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 672/1359] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:50.743641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:50.745151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:50.948736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049393aa-a08c-4e65-acac-9d047a270944 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:50.998027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:51.047381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:51.048003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:51.048288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:51.048567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:51.050140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:51.147006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:51.149268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73f907-580b-4528-8c6a-1e8cd500ba0d service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:51.170725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 688/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:53.551769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:53.552239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:53.552357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:53.553888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 673/1361] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:53.560057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efa050-e196-4d60-86c3-0fac9bac399f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:53.560492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efa050-e196-4d60-86c3-0fac9bac399f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:53.560732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efa050-e196-4d60-86c3-0fac9bac399f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:53.634390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-80efa050-e196-4d60-86c3-0fac9bac399f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:53.635537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-80efa050-e196-4d60-86c3-0fac9bac399f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:53.676325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:53.736273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 689/1362] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:54.039800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:54.041445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:54.274921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70857df-fa9e-4e56-8bfa-fdd4d5772165 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:54.290292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:54.429780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:54.430750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:54.431063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:54.431342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:54.432960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:54.557679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:54.559270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c80bfe-cab6-4b96-8dab-03625ba1b5da service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:54.581903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 674/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:56.923437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837b7b-5b7f-4a94-b153-e799807eff2c None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:56.924950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837b7b-5b7f-4a94-b153-e799807eff2c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:56.925388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837b7b-5b7f-4a94-b153-e799807eff2c None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:56.927634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 690/1364] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:56.933149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c6bd3-039c-4781-b6bb-17a3c639d482 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:56.933679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c6bd3-039c-4781-b6bb-17a3c639d482 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:56.933856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c6bd3-039c-4781-b6bb-17a3c639d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:56.952676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:57.022974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7a6c6bd3-039c-4781-b6bb-17a3c639d482 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:57.024217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7a6c6bd3-039c-4781-b6bb-17a3c639d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:57.135034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 675/1365] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:57.427718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:57.428177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:57.428543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:57.430034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 691/1366] 10.222.0.64 () {54 vars in 962 bytes} [Wed Aug 24 08:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:57.437400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e30f95-76c1-4983-ab86-3e075f7e3d1e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:57.437824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e30f95-76c1-4983-ab86-3e075f7e3d1e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:57.438263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e30f95-76c1-4983-ab86-3e075f7e3d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:57.544878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-53e30f95-76c1-4983-ab86-3e075f7e3d1e None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:57.547233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-53e30f95-76c1-4983-ab86-3e075f7e3d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:57.596260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:57.694831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 676/1367] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:57.702746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:57.707111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:57.904197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809394-f18b-43b7-a4cb-b9e61f43474e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:57.928551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:58.048634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:58.048634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:58.048634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:58.048634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:58.050176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:58.151339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:58.153394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fe464-e0a9-4b5d-b5a1-7f838c991d1b service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:58.175385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 692/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:58.863290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84712f-25ed-486a-983f-b4fe94690e04 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:58.863828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84712f-25ed-486a-983f-b4fe94690e04 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:58.863828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84712f-25ed-486a-983f-b4fe94690e04 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:58.865412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 677/1369] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:58.872059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ff0ec-3d1c-4428-a17b-23e429373c01 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:58.872059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ff0ec-3d1c-4428-a17b-23e429373c01 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:58.872059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ff0ec-3d1c-4428-a17b-23e429373c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:58.905418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:58.945349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8a0ff0ec-3d1c-4428-a17b-23e429373c01 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:58.946747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8a0ff0ec-3d1c-4428-a17b-23e429373c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:59.049542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 693/1370] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:59.645301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:59.645301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:59.645301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:59.648076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 678/1371] 10.222.0.64 () {54 vars in 964 bytes} [Wed Aug 24 08:48:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:59.651932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4dc7-b759-4eeb-9ae0-8ad6d03bd819 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:59.652206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4dc7-b759-4eeb-9ae0-8ad6d03bd819 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:59.652733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4dc7-b759-4eeb-9ae0-8ad6d03bd819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:59.733397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-12cb4dc7-b759-4eeb-9ae0-8ad6d03bd819 None None] MFA Rules not processed for user `9a67c22370884cefb7795cc9eb8b2f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:59.734595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-12cb4dc7-b759-4eeb-9ae0-8ad6d03bd819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:59.789919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:59.823468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 694/1372] 10.222.0.64 () {58 vars in 1061 bytes} [Wed Aug 24 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:59.831814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:59.833340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:59.954840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6efe09-bcf8-4ba0-98bf-c66382f8abb7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:59.999604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:00.043984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:00.044680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:00.045051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:00.045320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:00.047371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:00.174682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:00.176171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf034-184a-4599-9848-4fc5f9430128 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:00.202392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 679/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:00.366790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:00.367674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:00.367949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:00.369389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 695/1374] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:00.375434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1a124-323c-4a85-b96f-631a6ca9190d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:00.375942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1a124-323c-4a85-b96f-631a6ca9190d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:00.376220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1a124-323c-4a85-b96f-631a6ca9190d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:00.470501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-62d1a124-323c-4a85-b96f-631a6ca9190d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:00.472397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-62d1a124-323c-4a85-b96f-631a6ca9190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:00.494757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:00.580726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 680/1375] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:00.862863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:00.864394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:01.022251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5fd6-49c2-48d0-bcc3-21c93f7683eb None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:01.075625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:01.138015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:01.138538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:01.138858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:01.139130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:01.140730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:01.232530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:01.234266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5d844-196d-4eed-8e85-c312d0c39aa3 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:01.253009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 696/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:03.338039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:03.340215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:03.340215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:03.340215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 681/1377] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:03.348677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13a729-bfec-415a-a8b5-8319133040fc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:03.349156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13a729-bfec-415a-a8b5-8319133040fc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:03.349278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13a729-bfec-415a-a8b5-8319133040fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:03.417676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8a13a729-bfec-415a-a8b5-8319133040fc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:03.419027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8a13a729-bfec-415a-a8b5-8319133040fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:03.427660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:03.551349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 697/1378] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:03.848932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:03.850487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:04.048484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae72bf-f14b-40bb-bf92-0c168b6ac99f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:04.062118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:04.185569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:04.186189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:04.186470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:04.186774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:04.188494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:04.290876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:04.292600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3516a4a-787f-43e6-b73a-6f4c6b5a5a0c service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:04.313572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 682/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:06.383375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.383995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.383995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.385418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 698/1380] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:06.391084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9a0-f14b-4c80-859d-904911a2d7b1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.392605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9a0-f14b-4c80-859d-904911a2d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.393107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4f9a0-f14b-4c80-859d-904911a2d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.405912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:06.469723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-abd4f9a0-f14b-4c80-859d-904911a2d7b1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:06.471065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-abd4f9a0-f14b-4c80-859d-904911a2d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:06.600066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 683/1381] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:06.899559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:06.901236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:07.066400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:07.077359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26838778-69a7-42fe-ad2c-f5fe90c3027a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:07.216423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:07.217092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:07.217428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:07.217794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:07.219561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:07.326082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:07.327744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd2827-c213-4680-951b-f4569041c427 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:07.342343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:07.352780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 699/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:09.488747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.489460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.489460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.490668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 684/1383] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:09.496414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecc4d-1a35-438e-b6b0-2714be4c4727 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.496858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecc4d-1a35-438e-b6b0-2714be4c4727 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.497122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecc4d-1a35-438e-b6b0-2714be4c4727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.586502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8b4ecc4d-1a35-438e-b6b0-2714be4c4727 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:09.588061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8b4ecc4d-1a35-438e-b6b0-2714be4c4727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:09.699593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 700/1384] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:09.990920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:09.992882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.089922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:10.171208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a937f-357b-4a4e-b797-83a4f4f890f9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:10.294640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:10.295619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:10.295999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:10.296333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:10.299835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.364832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:10.411476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:10.413155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25133bc6-d609-47ce-adce-77ad14e68326 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:10.432832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 685/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:12.390847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:12.391422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:12.391422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:12.393513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 701/1386] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:12.399449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121d1a-d4dc-4e7a-9b7f-e3809a300eb9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:12.402463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121d1a-d4dc-4e7a-9b7f-e3809a300eb9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:12.402463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121d1a-d4dc-4e7a-9b7f-e3809a300eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:12.477933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-83121d1a-d4dc-4e7a-9b7f-e3809a300eb9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:12.479230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-83121d1a-d4dc-4e7a-9b7f-e3809a300eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:12.586962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 686/1387] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:12.876864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:12.878384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:12.979578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:13.071503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582582bb-fcf6-4a9a-a9f0-f277106b3232 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:13.193171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:13.193788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:13.194067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:13.194337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:13.195963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.239218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:13.298938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:13.301057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5a7d0-acaa-460c-a6bb-f2fe4b3c6551 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:13.325126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 702/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:15.427925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:15.428496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:15.428496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.429922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 687/1389] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:15.436137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8ef6-a1e9-41e0-8042-02c360664dfd None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:15.436707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8ef6-a1e9-41e0-8042-02c360664dfd None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:15.436707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8ef6-a1e9-41e0-8042-02c360664dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.548950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ef8d8ef6-a1e9-41e0-8042-02c360664dfd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:15.550191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ef8d8ef6-a1e9-41e0-8042-02c360664dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.658453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 703/1390] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:15.954720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:15.956265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.988159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.051423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f38e-3bbf-491f-812d-5d4e229beb31 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.051730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f38e-3bbf-491f-812d-5d4e229beb31 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.052032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f38e-3bbf-491f-812d-5d4e229beb31 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.053498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 704/1391] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:16.059779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c96-985f-4214-b76e-9606f969a17f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.060235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c96-985f-4214-b76e-9606f969a17f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.060495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c96-985f-4214-b76e-9606f969a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.080788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.115144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccec40-f47a-4d94-9135-5824d748ea54 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:16.136808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-20228c96-985f-4214-b76e-9606f969a17f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:16.138135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-20228c96-985f-4214-b76e-9606f969a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.237853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:16.238409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.238741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.239011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.241022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.242317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 705/1392] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:16.297260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.343780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:16.345445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e81ff-5c99-4dfa-b65f-ff18131d0511 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:16.365539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 688/1393] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:16.554804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.555381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.555381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.556682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 706/1394] 10.222.0.64 () {54 vars in 967 bytes} [Wed Aug 24 08:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:16.561892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180100c-c023-447e-af8e-1d23bdf47aa6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.562582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180100c-c023-447e-af8e-1d23bdf47aa6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.562582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180100c-c023-447e-af8e-1d23bdf47aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.644042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c180100c-c023-447e-af8e-1d23bdf47aa6 None None] MFA Rules not processed for user `d94c7f51f5d5478f858a8eb47cd89257`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:16.645292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c180100c-c023-447e-af8e-1d23bdf47aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.703077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.733061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 689/1395] 10.222.0.64 () {58 vars in 1064 bytes} [Wed Aug 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:16.742198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:16.747977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.855162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.862747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60ad62-fdac-46d3-ad66-19838ed81806 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:16.945588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:16.945588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.945832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.946043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.948375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.067370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:17.068823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984cd4f-5773-4cef-88d4-e7dec1df0f23 service glance] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:17.088975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 707/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:17.962196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb149c-d6ad-433b-a34a-6ebaafebcf49 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:17.962768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb149c-d6ad-433b-a34a-6ebaafebcf49 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:17.962874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb149c-d6ad-433b-a34a-6ebaafebcf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.036351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-38cb149c-d6ad-433b-a34a-6ebaafebcf49 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:18.037562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-38cb149c-d6ad-433b-a34a-6ebaafebcf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.128388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 690/1397] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:49:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:18.172210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.173252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.173252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.174629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 708/1398] 10.222.0.64 () {54 vars in 943 bytes} [Wed Aug 24 08:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:18.181242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b5efc-7826-44c4-ba75-b80253bae042 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.181527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b5efc-7826-44c4-ba75-b80253bae042 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.181801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b5efc-7826-44c4-ba75-b80253bae042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.263951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-706b5efc-7826-44c4-ba75-b80253bae042 None None] MFA Rules not processed for user `a3b614a59a2e40c09dc87e99a717caf3`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:18.265104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-706b5efc-7826-44c4-ba75-b80253bae042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.279633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.381549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 691/1399] 10.222.0.64 () {58 vars in 1040 bytes} [Wed Aug 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:18.391268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:18.392921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.419335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.542461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe97d4f-d3cd-47cb-862f-cb8b97ad151f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:18.640454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a7dd-74eb-456c-a90a-9672695426df None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.640814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a7dd-74eb-456c-a90a-9672695426df None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.641022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a7dd-74eb-456c-a90a-9672695426df None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.642397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 692/1400] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:18.648210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50931b52-117a-4769-8b0d-51735021eb22 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.648538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50931b52-117a-4769-8b0d-51735021eb22 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.648971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50931b52-117a-4769-8b0d-51735021eb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.653827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:18.654450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.654765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.655043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.656752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.664562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.735421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-50931b52-117a-4769-8b0d-51735021eb22 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:18.736664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-50931b52-117a-4769-8b0d-51735021eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.747493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:18.749111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:18.758152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 709/1401] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:49:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:49:18.818710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.842392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 693/1402] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:19.142486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:19.143869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:19.243813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79213006-5b5a-41f3-8044-317997d080a3 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:19.303607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:19.395672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:19.396568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:19.397089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:19.397556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:19.400441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:19.507117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:19.508736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1449947-1ee1-4c8b-a486-f43d75231a6a service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:19.530432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 710/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:21.628435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:21.628435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:21.628435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:21.628435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 694/1404] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:21.633615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec593-b85d-4be3-9e4c-2b88dd871a90 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:21.634087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec593-b85d-4be3-9e4c-2b88dd871a90 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:21.634087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec593-b85d-4be3-9e4c-2b88dd871a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:21.716246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-114ec593-b85d-4be3-9e4c-2b88dd871a90 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:21.717601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-114ec593-b85d-4be3-9e4c-2b88dd871a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.743664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:21.853579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 711/1405] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:22.174242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:22.175805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:22.359666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e8759-e30b-415e-8813-38d2c3684f8e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:22.432304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:22.493909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:22.494529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:22.494860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:22.495134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:22.496772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:22.610942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:22.612765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6f5c-a954-4248-815d-f93fc33b4a73 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:22.632751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 695/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:24.759621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:24.760189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:24.760189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:24.761544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 712/1407] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:24.767954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a558b-c14d-4ec7-9d75-81a8c7c603aa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:24.768501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a558b-c14d-4ec7-9d75-81a8c7c603aa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:24.768609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a558b-c14d-4ec7-9d75-81a8c7c603aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:24.845624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-033a558b-c14d-4ec7-9d75-81a8c7c603aa None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:24.846904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-033a558b-c14d-4ec7-9d75-81a8c7c603aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:24.868534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:24.961687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 696/1408] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:25.255278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:25.256820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:25.471509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:25.474466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738869d-e0bc-44c4-b1c9-f2b099f014eb None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:25.622493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:25.623202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:25.623479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:25.623750 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:25.625946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:25.745516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:25.747191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ffcd-e792-4305-894e-d28a4124baaf service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:25.766372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 713/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:34.958576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=35 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:35.763912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:43.282677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2acff-e835-46d9-9d83-1ecaffb7e43c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:43.283237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2acff-e835-46d9-9d83-1ecaffb7e43c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:43.283237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2acff-e835-46d9-9d83-1ecaffb7e43c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:43.284591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 697/1410] 10.222.0.64 () {54 vars in 918 bytes} [Wed Aug 24 08:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:43.289901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6956f-d5d9-4e5e-af1b-0fef0a97b505 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:43.290350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6956f-d5d9-4e5e-af1b-0fef0a97b505 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:43.290457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6956f-d5d9-4e5e-af1b-0fef0a97b505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:43.367196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-fdf6956f-d5d9-4e5e-af1b-0fef0a97b505 None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:43.368429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fdf6956f-d5d9-4e5e-af1b-0fef0a97b505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:43.477540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 714/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:43.493023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb1d89-8c89-4647-a63b-0f7e37796aff None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:43.493311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb1d89-8c89-4647-a63b-0f7e37796aff None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:43.493593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb1d89-8c89-4647-a63b-0f7e37796aff None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:43.495048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 698/1412] 10.222.0.64 () {54 vars in 918 bytes} [Wed Aug 24 08:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:43.500081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003b02-b777-4c82-a4e3-b2793670074c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:43.500372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003b02-b777-4c82-a4e3-b2793670074c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:43.500646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003b02-b777-4c82-a4e3-b2793670074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:43.574542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:43.578089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-19003b02-b777-4c82-a4e3-b2793670074c None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:43.579322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-19003b02-b777-4c82-a4e3-b2793670074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:43.698411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 715/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:46.086953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:46.087497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:46.087497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:46.088884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:46.094125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561069bf-c3ad-41af-925a-f4d036b042de None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:46.094536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561069bf-c3ad-41af-925a-f4d036b042de None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:46.094700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561069bf-c3ad-41af-925a-f4d036b042de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:46.161755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-561069bf-c3ad-41af-925a-f4d036b042de None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:46.162998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-561069bf-c3ad-41af-925a-f4d036b042de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:46.218235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:46.265189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:46.553774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:46.555391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:46.701502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5b516-7a20-439f-a17a-5de2d46df579 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:46.774813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:46.816404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:46.817014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:46.817290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:46.817563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:46.819213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:46.912025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:46.913668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a1b7-d437-4c76-9ea0-eb492d97367a service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:46.931995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 700/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:47.849631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.850261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.850261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.851527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:47.856783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070c18-3c4d-46b6-94f4-01ff4a63ea55 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.857262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070c18-3c4d-46b6-94f4-01ff4a63ea55 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.857368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070c18-3c4d-46b6-94f4-01ff4a63ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.938964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-00070c18-3c4d-46b6-94f4-01ff4a63ea55 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:47.940416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-00070c18-3c4d-46b6-94f4-01ff4a63ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.971407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:48.050762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:48.202817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:48.204344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:48.355820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65a2e4-322b-4ece-bdf7-f6c53c370a14 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:48.402188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:48.475708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:48.476235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:48.476510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:48.476809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:48.478406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:48.567997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:48.569640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d455307-0587-4332-9cde-4ac6039595b2 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:48.587776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 718/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:49.476902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:49.477484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:49.477484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:49.478909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:49.484668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cea8-5c1c-4c58-9e54-aea36ea9a781 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:49.485139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cea8-5c1c-4c58-9e54-aea36ea9a781 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:49.485237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cea8-5c1c-4c58-9e54-aea36ea9a781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:49.539883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:49.563220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c803cea8-5c1c-4c58-9e54-aea36ea9a781 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:49.564463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c803cea8-5c1c-4c58-9e54-aea36ea9a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:49.662504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:49.813425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:49.814959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:49.966502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b69bcb-d2ea-48b4-9e58-f30a83005f4a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:49.978367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:50.081701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:50.082222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:50.082494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:50.082785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:50.084380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:50.173697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:50.175360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345e64-e8eb-4645-8476-363dce09f95e service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:50.194703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 703/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:51.094722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:51.095306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:51.095306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:51.096644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:51.101690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5cf1e-f68e-4738-9060-8518db8f5140 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:51.102175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5cf1e-f68e-4738-9060-8518db8f5140 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:51.102287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5cf1e-f68e-4738-9060-8518db8f5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:51.121524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:51.177293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-85d5cf1e-f68e-4738-9060-8518db8f5140 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:51.178508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-85d5cf1e-f68e-4738-9060-8518db8f5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:51.279539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:51.430952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:51.432388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:51.555116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:51.593600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fe716-29c2-4e69-9c83-50d914b1aa82 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:51.706109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:51.706639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:51.706947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:51.707218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:51.708836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:51.783400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:51.797041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:51.798641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36628325-469a-47f6-973c-4b9df5d68ecd service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:51.817963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 721/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:52.684280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:52.684876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:52.684876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:52.686936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:52.692236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790ac7-0e5b-4ba0-8fda-600c7040e95f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:52.692754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790ac7-0e5b-4ba0-8fda-600c7040e95f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:52.692754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790ac7-0e5b-4ba0-8fda-600c7040e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:52.768995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7c790ac7-0e5b-4ba0-8fda-600c7040e95f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:52.770201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7c790ac7-0e5b-4ba0-8fda-600c7040e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:52.865153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:53.015688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:53.017182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:53.103564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:53.169092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5e917-aa8f-4612-be70-a6304ee1525c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:53.282780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:53.283391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:53.283665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:53.283934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:53.285534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:53.337502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:53.380579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:53.382205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ddc33-38a3-43a0-8d91-27cc900a5960 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:53.401000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 706/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:49:54.297879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:54.298478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:54.298478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:54.299817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:54.305136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fba910-3aae-4ad0-8c3b-bbb9adbfd33f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:54.305616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fba910-3aae-4ad0-8c3b-bbb9adbfd33f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:54.305716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fba910-3aae-4ad0-8c3b-bbb9adbfd33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:54.384301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-92fba910-3aae-4ad0-8c3b-bbb9adbfd33f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:54.385534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-92fba910-3aae-4ad0-8c3b-bbb9adbfd33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:54.485773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:54.636591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:54.638021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:54.687565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:54.788501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af54d-02fb-4e3a-ab74-727d5172df6f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:54.899175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:54.899748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:54.900034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:54.900310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:54.902166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:54.909233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:54.991800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:54.993397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb888f3-1411-43e9-aebb-40d5de09fc4d service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:55.010975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 724/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:55.862715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:55.863277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:55.863277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:55.864624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:55.869746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a37a1-899a-4068-b65d-235fc0d3f992 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:55.870158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a37a1-899a-4068-b65d-235fc0d3f992 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:55.870294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a37a1-899a-4068-b65d-235fc0d3f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:55.945634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-582a37a1-899a-4068-b65d-235fc0d3f992 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:55.946870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-582a37a1-899a-4068-b65d-235fc0d3f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:56.015020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:56.044082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:56.194291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:56.195856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:56.213331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:56.342508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dece8-128d-4ab1-b6d4-dcb5386acf35 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:56.431199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:56.456962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:56.457507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:56.457782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:56.458054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:56.459702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:56.552525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:56.554181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f2310-2357-4364-a1e4-d67d86dda82b service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:56.572405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 709/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:57.438808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:57.439558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:57.439558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:57.440748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:57.445714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0182eb-7d8e-4e02-874f-247cd998bf8f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:57.452077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0182eb-7d8e-4e02-874f-247cd998bf8f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:57.452077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0182eb-7d8e-4e02-874f-247cd998bf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:57.523894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4c0182eb-7d8e-4e02-874f-247cd998bf8f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:57.525132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4c0182eb-7d8e-4e02-874f-247cd998bf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:57.569559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:57.626084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:57.778751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:57.780707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:57.928936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c8df2-dc03-44f1-9e88-1c706c9681fa None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:57.985379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:58.057096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:58.057616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:58.057903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:58.058181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:58.059818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:58.151809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:58.153435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd4373-0ea3-436a-bc4f-cb640d0fb219 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:58.172372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 727/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:59.035219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:59.035864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:59.035864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:59.037106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:59.042047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560f7-aace-4d24-82a7-0c3604ff2a6e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:59.042522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560f7-aace-4d24-82a7-0c3604ff2a6e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:59.042623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560f7-aace-4d24-82a7-0c3604ff2a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:59.123743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7a8560f7-aace-4d24-82a7-0c3604ff2a6e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:59.124952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7a8560f7-aace-4d24-82a7-0c3604ff2a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:59.127006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:59.226350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:59.377287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:59.378851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:59.529104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc6722-026a-4aa2-ab26-2db478bb868e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:59.559182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:59.645462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:59.646074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:59.646354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:59.646625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:59.649140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:59.739636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:59.741277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c0a2c-ccb0-4065-b97d-5bee00ae9053 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:59.759295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 712/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:01.612228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:01.612802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:01.612915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:01.614203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:01.619729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadc6e-8b60-47ed-812f-25846ddcb2f2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:01.620181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadc6e-8b60-47ed-812f-25846ddcb2f2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:01.620282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadc6e-8b60-47ed-812f-25846ddcb2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:01.661337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:01.686814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-67eadc6e-8b60-47ed-812f-25846ddcb2f2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:01.688036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-67eadc6e-8b60-47ed-812f-25846ddcb2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:01.788014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:02.079167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:02.080693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:02.184459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02672ffe-a437-44f8-aae7-514bf191d77e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:02.342500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:02.384295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:02.384914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:02.385187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:02.385479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:02.387843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:02.501514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:02.502978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ee248-abd8-4d32-ae50-823eca2f7e3c service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:02.521294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 730/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:04.603986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:04.604528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:04.604646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:04.605913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:04.611486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a81ddc-2b51-420c-9e34-2a7bdbb06dbb None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:04.611936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a81ddc-2b51-420c-9e34-2a7bdbb06dbb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:04.612034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a81ddc-2b51-420c-9e34-2a7bdbb06dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:04.679326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b1a81ddc-2b51-420c-9e34-2a7bdbb06dbb None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:04.680559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b1a81ddc-2b51-420c-9e34-2a7bdbb06dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:04.694088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:04.779590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:05.058845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:05.060958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:05.164554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf08d8-8992-4c04-83c7-d0fbc64c0b4c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:05.177220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:05.244358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:05.244969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:05.245252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:05.245525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:05.247221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:05.378538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:05.379937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f5212-8988-4428-bbef-d4ced91c4df6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:05.397937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 715/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:07.775873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:07.776659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:07.776659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:07.777927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:07.783858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1968f-3397-43be-925b-7d80171b18e5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:07.784280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1968f-3397-43be-925b-7d80171b18e5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:07.784417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1968f-3397-43be-925b-7d80171b18e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:07.833142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:07.850862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6aa1968f-3397-43be-925b-7d80171b18e5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:07.852080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6aa1968f-3397-43be-925b-7d80171b18e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:07.950602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:08.237012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:08.238542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:08.353563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7495a77-4783-4aac-a2fd-360c7c37fc05 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:08.396975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:08.444004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:08.444621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:08.444900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:08.445174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:08.446835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:08.571145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:08.572586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcbc07-be56-48ec-9982-27f39d229623 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:08.592457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 733/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:10.638790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.639340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.639340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.640679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:10.646199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48dc8-fdba-431c-92d4-25ae25e93ead None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.646721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48dc8-fdba-431c-92d4-25ae25e93ead None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.646825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48dc8-fdba-431c-92d4-25ae25e93ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.717076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9cb48dc8-fdba-431c-92d4-25ae25e93ead None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:10.718294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9cb48dc8-fdba-431c-92d4-25ae25e93ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:10.740294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:10.821342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:11.112506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:11.114041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.222454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497ddc1-6e16-42ee-ae83-1c4b0867c7f9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:11.242945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:11.305406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:11.306077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:11.306406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:11.306772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:11.308572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.429386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:11.430891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd652f-60fa-45bd-95bb-36d8c5564444 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:11.451318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 718/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:13.444259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503408-6c46-4051-982c-a265fb59b807 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.445018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503408-6c46-4051-982c-a265fb59b807 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.445018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503408-6c46-4051-982c-a265fb59b807 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.446290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:13.457373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc10786-f323-4054-a0ea-01df89748df7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.457874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc10786-f323-4054-a0ea-01df89748df7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.457974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc10786-f323-4054-a0ea-01df89748df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.522280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:13.525369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3bc10786-f323-4054-a0ea-01df89748df7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:13.526894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3bc10786-f323-4054-a0ea-01df89748df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:13.624239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:13.908460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503408-6c46-4051-982c-a265fb59b807 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:13.910051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-21503408-6c46-4051-982c-a265fb59b807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:14.014414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503408-6c46-4051-982c-a265fb59b807 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:14.065085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:14.091868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:14.092482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:14.092762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:14.093033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:14.094672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:14.209732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:14.211299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927d363-5692-4192-8395-f9ead83233a3 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:14.231048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 736/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:16.233824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.234404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.234404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.235790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:16.241130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868f24-36e0-4470-a930-2412640c8bf0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.241593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868f24-36e0-4470-a930-2412640c8bf0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.241701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868f24-36e0-4470-a930-2412640c8bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.311357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1c868f24-36e0-4470-a930-2412640c8bf0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:16.312582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1c868f24-36e0-4470-a930-2412640c8bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.343140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:16.412898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:16.700558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:16.702108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.809180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804ebb1-8134-43c0-852d-1bae6519027f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:16.838415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:16.887382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:16.888005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.888289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.888565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.890242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:17.007416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:17.008882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec27ef1-e652-4705-8141-3dc0b4c2bd72 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:17.029062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 721/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:19.030676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.031285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.031285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.032591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:19.038073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a9e3-93a8-473c-b1e4-e2faa4e461a0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.038527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a9e3-93a8-473c-b1e4-e2faa4e461a0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.038680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5a9e3-93a8-473c-b1e4-e2faa4e461a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.107250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5be5a9e3-93a8-473c-b1e4-e2faa4e461a0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:19.108507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5be5a9e3-93a8-473c-b1e4-e2faa4e461a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.117315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:19.209596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:19.495250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:19.496793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.602770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c0582-f753-46f7-945d-18a213e4f73e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:19.659920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:19.679225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:19.679837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.680113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.680381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.681971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.793070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:19.794492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad17cb-c3b8-412f-9e6c-554c6c283a4a service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:19.814753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 739/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:21.782644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:21.783258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:21.783258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:21.784503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:21.789885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e537b-36d1-40b9-bf2b-a92ba0f8e747 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:21.790359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e537b-36d1-40b9-bf2b-a92ba0f8e747 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:21.790475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e537b-36d1-40b9-bf2b-a92ba0f8e747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:21.856303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8d1e537b-36d1-40b9-bf2b-a92ba0f8e747 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:21.858019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8d1e537b-36d1-40b9-bf2b-a92ba0f8e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:21.896440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:21.957817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:22.243761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:22.245318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.351294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b86fff-7107-40a1-8295-b930ee5f6924 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:22.390049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:22.428438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:22.429062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:22.429584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:22.430191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:22.431895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.549363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:22.550794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce51ee-0500-4642-b66f-7f3c938ff569 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:22.570954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 724/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:24.538416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:24.539079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:24.539079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:24.540339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:24.546010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b238c-ca22-4856-aaac-df2e3f67fd4a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:24.546491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b238c-ca22-4856-aaac-df2e3f67fd4a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:24.546612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b238c-ca22-4856-aaac-df2e3f67fd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:24.614198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5d0b238c-ca22-4856-aaac-df2e3f67fd4a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:24.615474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5d0b238c-ca22-4856-aaac-df2e3f67fd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:24.634104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:24.713365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:24.993080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:24.994597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.100779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e81b1-21ac-42c0-b174-e865cdc8c5df None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:25.171350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:25.178250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:25.179319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.179815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.180314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.183299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.298115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:25.299601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09235d3f-f78c-43b8-b24d-847b163ef026 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:25.317889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 742/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:27.255568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:27.256148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:27.256148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:27.257516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:27.263297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e9a6d-2a61-4c5e-b2ab-271c817785ff None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:27.263812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e9a6d-2a61-4c5e-b2ab-271c817785ff None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:27.263924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e9a6d-2a61-4c5e-b2ab-271c817785ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:27.336435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5c3e9a6d-2a61-4c5e-b2ab-271c817785ff None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:27.337755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5c3e9a6d-2a61-4c5e-b2ab-271c817785ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:27.392430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:27.445705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:27.739288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:27.740918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:27.858620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd604a-7cf5-4caf-947f-e42939b96ca6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:27.923129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:27.952583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:27.953203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:27.953480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:27.953748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:27.955533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:28.081766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:28.083304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e7e64-f33d-496d-8533-fdc10ae1d70c service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:28.113818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 727/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:37.443834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=22 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:38.109949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=66 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:47.849431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ed84-8c7e-454e-a16e-9ba82d265ea5 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:47.850034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ed84-8c7e-454e-a16e-9ba82d265ea5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:47.850034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ed84-8c7e-454e-a16e-9ba82d265ea5 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:47.851305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 744/1471] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:47.857596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea4860-7b3c-4c18-ada3-c065edea0f45 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:47.858124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea4860-7b3c-4c18-ada3-c065edea0f45 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:47.858124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea4860-7b3c-4c18-ada3-c065edea0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:47.927425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1aea4860-7b3c-4c18-ada3-c065edea0f45 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:47.928666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1aea4860-7b3c-4c18-ada3-c065edea0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:48.028690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 728/1472] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:50:48.338099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:48.338406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:48.338698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:48.340542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 745/1473] 10.222.0.64 () {54 vars in 967 bytes} [Wed Aug 24 08:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:48.345629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5660a-1abd-40db-86fa-821d009195a0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:48.345910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5660a-1abd-40db-86fa-821d009195a0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:48.346182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5660a-1abd-40db-86fa-821d009195a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:48.423838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-85d5660a-1abd-40db-86fa-821d009195a0 None None] MFA Rules not processed for user `d94c7f51f5d5478f858a8eb47cd89257`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:48.425054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-85d5660a-1abd-40db-86fa-821d009195a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:48.446175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:48.503420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 729/1474] 10.222.0.64 () {58 vars in 1064 bytes} [Wed Aug 24 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:48.510961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:48.512450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:48.617068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ae5d4-ef53-4784-88b1-5912156b9c95 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:48.690099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:48.690804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:48.691090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:48.691365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:48.692959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:48.745902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:48.802563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:48.804059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d2759-7137-4628-990c-1c3018ac7a56 service glance] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:48.823105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 746/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:50.812090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:50.812807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:50.812807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:50.813945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 730/1476] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:50.819134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd160-138d-4499-ab65-3359fc8bac2b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:50.820456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd160-138d-4499-ab65-3359fc8bac2b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:50.820456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd160-138d-4499-ab65-3359fc8bac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:50.886548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-fc9bd160-138d-4499-ab65-3359fc8bac2b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:50.887877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fc9bd160-138d-4499-ab65-3359fc8bac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:50.985614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 747/1477] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:51.263157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:51.264680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:51.391447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702904b-7fcc-4f31-a6d5-c18c6b16e9bd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:51.442625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:51.489037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:51.489648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:51.489931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:51.490203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:51.490880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:51.492731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:50:51.492731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:50:51.492731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:50:51.493891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab64c2-c6d2-486d-b482-fd5a7aaa5aa7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:51.513059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 731/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Aug 24 08:50:51 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:50:53.479868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:53.480483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:53.480483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:53.482333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 748/1479] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:53.488013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5936-7c4a-4e7f-8469-fa15456f49fd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:53.488482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5936-7c4a-4e7f-8469-fa15456f49fd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:53.488600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5936-7c4a-4e7f-8469-fa15456f49fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:53.555655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9feb5936-7c4a-4e7f-8469-fa15456f49fd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:53.556884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9feb5936-7c4a-4e7f-8469-fa15456f49fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:53.655086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 732/1480] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:53.955554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:53.957104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:53.967596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:54.109802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537f010-0670-4e0a-b9cb-c3027f3dde82 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:54.190406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:54.224308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:54.224839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:54.225122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:54.225395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:54.227078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:54.316644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:54.318262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062406fe-cf16-4a92-b566-e6734b30443e service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:54.337116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 749/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:56.469492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:56.470080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:56.470080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:56.471468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 733/1482] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:56.476918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c27d7-226a-424f-a61c-6a72a6540332 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:56.477495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c27d7-226a-424f-a61c-6a72a6540332 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:56.477495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c27d7-226a-424f-a61c-6a72a6540332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:56.545016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5c0c27d7-226a-424f-a61c-6a72a6540332 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:56.546227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5c0c27d7-226a-424f-a61c-6a72a6540332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:56.592829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:56.645266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 750/1483] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:56.931551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:56.933147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:56.957148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:57.037763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c9c9e-77ab-4043-90e5-9f2d3edab083 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:57.112431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:57.113044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:57.113326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:57.113596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:57.115244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:57.196571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:57.233256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:57.234748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0c42-a11b-4bf2-92ea-538af508a23b service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:57.370336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 734/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:59.362556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:59.363157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:59.363157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:59.364497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 751/1485] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:59.370008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933fcbf-1089-408f-8e1d-7a2015b4760b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:59.370456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933fcbf-1089-408f-8e1d-7a2015b4760b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:59.370580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933fcbf-1089-408f-8e1d-7a2015b4760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:59.437436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8933fcbf-1089-408f-8e1d-7a2015b4760b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:59.438711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8933fcbf-1089-408f-8e1d-7a2015b4760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:59.536830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 735/1486] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:59.822629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:59.824735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:59.926728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d566f-f71d-4282-96f5-4289c2d05f90 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:59.998778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:00.000569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:00.001194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:00.001478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:00.001750 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:00.003393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:00.114245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:00.115647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba446c9b-1880-45f4-be8c-229d9b7564d3 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:00.135303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 752/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:51:02.278580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:02.279176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:02.279176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:02.280601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 736/1488] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:02.286552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211d3-0205-44d8-8652-bc246b89da66 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:02.287049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211d3-0205-44d8-8652-bc246b89da66 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:02.287182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211d3-0205-44d8-8652-bc246b89da66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:02.354413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8c0211d3-0205-44d8-8652-bc246b89da66 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:02.356178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8c0211d3-0205-44d8-8652-bc246b89da66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:02.409664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:02.454731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 753/1489] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:02.750275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:02.751847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:02.788273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:02.857258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76b36-bb6c-4cde-9dc9-1784fbde689c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:02.931335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:02.931948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:02.932231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:02.932505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:02.934216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:03.017448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:03.053276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:03.054781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c086a1d-114a-4411-abd5-21baf4dedea6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:03.074611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 737/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:51:05.434770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:05.435355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:05.435355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:05.436695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 754/1491] 10.222.0.64 () {54 vars in 927 bytes} [Wed Aug 24 08:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:05.442304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867838-ee3a-423b-b6f5-e50c27fecdbd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:05.442874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867838-ee3a-423b-b6f5-e50c27fecdbd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:05.442874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867838-ee3a-423b-b6f5-e50c27fecdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:05.510082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5d867838-ee3a-423b-b6f5-e50c27fecdbd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:05.511381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5d867838-ee3a-423b-b6f5-e50c27fecdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:05.609504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 738/1492] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Aug 24 08:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:05.900324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:05.901850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:06.006860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9e384-8a4f-4661-9db6-3255c3d4c742 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:06.086061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:06.086703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:06.086986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:06.087256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:06.088869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:06.099534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:06.208497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:06.209960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16d78c-ea40-4503-be14-72f33da48c98 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:06.233349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 755/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:51:15.607860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=82 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:16.228380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=50 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:27.330801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:27.331377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:27.331377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:27.406414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:52:27.407671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:27.503009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 739/1494] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:52:27.511832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:27.512370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:27.512476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:27.594891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:52:27.596092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:27.697503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 756/1495] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:52:27.702263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:27.703519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:27.714865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6486db-efb8-45a7-8a42-f26b1fe689b5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:27.777052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:27.832342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:27.832342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:27.832342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:27.832342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:27.832839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 740/1496] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:52:27.839464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:27.849412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e397-0ae0-4f6c-a62b-46ef84436de0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:27.909025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:27.942074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:27.942706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:27.942993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:27.943256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:27.943875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:27.945602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:27.945602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:52:27.945602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:52:27.946819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:28.082991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 757/1497] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:52:27 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:52:28.087123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:28.088691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:28.122764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c843721-8a65-4193-9ce5-2b8cd38b60a8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:28.215423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:28.216033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:28.216306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:28.216574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:28.217209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:28.218233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:28.218233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:52:28.218233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:52:28.219474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:28.325715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 741/1498] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:52:28 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:52:28.329962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:28.331530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:28.393792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd32fb3-513f-4782-837b-3d96dd1ed10c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:28.475025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:28.485331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:52:28.490836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:52:28.491543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:28.501150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 758/1499] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:52:28.505388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:28.506915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:28.518803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52303f-1d47-47ae-a050-4a65da1f6932 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:28.544927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:28.615953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:28.616563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:28.616842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:28.617116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:28.617764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:28.618850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:28.618850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:52:28.618850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:52:28.619745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4529d2-a925-478a-ac71-8968e8d88112 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:28.632226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 742/1500] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:52:28.636939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:28.638486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:28.671554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b6538-583e-4806-93f4-f867462665ac admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:28.764169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:28.764828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:28.765107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:28.765382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] PATH_INFO: `/v3/projects/e12d63b808b24e79892a5a5a27ea93e6/users/31e417c0747b4280b08874b7d138fcf6/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:28.787391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12d63b808b24e79892a5a5a27ea93e6, user_id=31e417c0747b4280b08874b7d138fcf6, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:28.788419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:28.788419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:52:28.788419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:52:28.790334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b30570-5c5c-4287-97cd-3a424b8f04fe admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:28.862296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 759/1501] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:52:28 2022] PUT /identity/v3/projects/e12d63b808b24e79892a5a5a27ea93e6/users/31e417c0747b4280b08874b7d138fcf6/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:52:28.875148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f065b-ca79-464d-a08a-996c83b7264a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:28.875490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f065b-ca79-464d-a08a-996c83b7264a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:28.875816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f065b-ca79-464d-a08a-996c83b7264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:28.932564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-057f065b-ca79-464d-a08a-996c83b7264a None None] MFA Rules not processed for user `31e417c0747b4280b08874b7d138fcf6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:52:28.934222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-057f065b-ca79-464d-a08a-996c83b7264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:29.010791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 743/1502] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:52:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:52:29.024728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:29.025021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:29.025302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:29.026668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 760/1503] 10.222.0.64 () {54 vars in 964 bytes} [Wed Aug 24 08:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:52:29.031827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e09e-dfe5-46ce-a838-3983f88a2d4a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:29.032113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e09e-dfe5-46ce-a838-3983f88a2d4a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:29.032384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e09e-dfe5-46ce-a838-3983f88a2d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:29.062792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:29.107940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2c08e09e-dfe5-46ce-a838-3983f88a2d4a None None] MFA Rules not processed for user `9a67c22370884cefb7795cc9eb8b2f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:52:29.109141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c08e09e-dfe5-46ce-a838-3983f88a2d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:29.182870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 744/1504] 10.222.0.64 () {58 vars in 1061 bytes} [Wed Aug 24 08:52:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:52:29.191059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:29.192590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:29.242636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:29.295715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28b79-e7da-4a64-849c-719fb51ee76e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:29.372828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:29.373449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:29.373745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:29.374021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:29.375693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:29.479661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:29.481681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:29.485213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:29.499401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 761/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:52:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:52:29.545094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a061f-21c9-4f55-ae87-784809da99f8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:29.545561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a061f-21c9-4f55-ae87-784809da99f8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:29.545673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a061f-21c9-4f55-ae87-784809da99f8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:29.546959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 745/1506] 10.222.0.64 () {52 vars in 830 bytes} [Wed Aug 24 08:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:52:29.550691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:29.551884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:29.649685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbae0c-0980-4489-b43a-8ddc7f651c4c service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:29.741948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:29.742498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:29.742878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:29.743147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] PATH_INFO: `/v3/users/31e417c0747b4280b08874b7d138fcf6` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:29.755006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e417c0747b4280b08874b7d138fcf6)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:29.756026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:29.756026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:52:29.756026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:52:29.757235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:29.766607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:29.987386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e417c0747b4280b08874b7d138fcf6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:29.987819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e417c0747b4280b08874b7d138fcf6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:30.067817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e417c0747b4280b08874b7d138fcf6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:30.082025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e417c0747b4280b08874b7d138fcf6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:30.104166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e417c0747b4280b08874b7d138fcf6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:30.110430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 762/1507] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:52:29 2022] DELETE /identity/v3/users/31e417c0747b4280b08874b7d138fcf6 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:52:30.114870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:30.115139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:30.115408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:30.180372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:52:30.181614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:30.257942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:30.270715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 746/1508] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:52:30.281754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:30.283054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:30.388822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4dec-2e22-4303-869b-ba475e11ca56 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:30.446241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:30.466662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:30.467302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:30.467581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:30.467848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:30.469538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:30.577001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:30.578446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37f1d-af7c-4e6d-bc10-1f6b059d5980 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:30.596501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 763/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:52:31.115515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:52:31.116808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:52:31.128351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d2480-66a9-4d39-819c-ab729829c0ab None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:52:31.224629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:52:31.225179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:52:31.225458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:52:31.225727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] PATH_INFO: `/v3/projects/e12d63b808b24e79892a5a5a27ea93e6` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:52:31.236184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12d63b808b24e79892a5a5a27ea93e6)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:52:31.237210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:52:31.237210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:52:31.237210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:52:31.238408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:52:31.549464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12d63b808b24e79892a5a5a27ea93e6', 'request_id': 'req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:31.555473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12d63b808b24e79892a5a5a27ea93e6', 'request_id': 'req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:31.650790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12d63b808b24e79892a5a5a27ea93e6', 'request_id': 'req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:31.684265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] The token cache is being invalidate because project e12d63b808b24e79892a5a5a27ea93e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:52:31.684551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:31.685073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b3e8a5d-7b74-4a2e-b7e9-626b07f5fa5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:52:31.686410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 747/1510] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:52:31 2022] DELETE /identity/v3/projects/e12d63b808b24e79892a5a5a27ea93e6 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:52:40.824270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=62 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:41.681560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:01.692448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:01.693050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:01.693050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:01.694394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:01.699222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a511c-37b1-4342-b041-2e93c6fa68a3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:01.699637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a511c-37b1-4342-b041-2e93c6fa68a3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:01.699782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a511c-37b1-4342-b041-2e93c6fa68a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:01.778269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f13a511c-37b1-4342-b041-2e93c6fa68a3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:01.779445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f13a511c-37b1-4342-b041-2e93c6fa68a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:01.882524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:02.038494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:02.040694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:02.160597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418088b-5bef-4724-800b-47da8bfde8c4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:02.166164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:02.166834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:02.167117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:02.167390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:02.175722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:02.177318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155eaa-394d-4215-b111-75323136dad7 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:02.187019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 765/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:03.034945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084d09-1756-47a6-8fa4-501f56c93586 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:03.035866 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084d09-1756-47a6-8fa4-501f56c93586 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:03.035866 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084d09-1756-47a6-8fa4-501f56c93586 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:03.037203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:03.042314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22315-4de8-4efb-8026-5ac994c409af None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:03.042820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22315-4de8-4efb-8026-5ac994c409af None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:03.042932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22315-4de8-4efb-8026-5ac994c409af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:03.064372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-0ae22315-4de8-4efb-8026-5ac994c409af None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:03.065660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0ae22315-4de8-4efb-8026-5ac994c409af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:03.073900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:03.231717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22084d09-1756-47a6-8fa4-501f56c93586 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:03.246267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22084d09-1756-47a6-8fa4-501f56c93586 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:03.252046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:03.252676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:03.252981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:03.253259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:03.261608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:03.263190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ced69-4803-4db6-aef0-877b34764057 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:03.272548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 750/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:03.662983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a633-8216-4f50-a123-9e673adab031 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:03.663306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a633-8216-4f50-a123-9e673adab031 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:03.663589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a633-8216-4f50-a123-9e673adab031 None None] PATH_INFO: `/v3` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:03.665006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 767/1517] 10.222.0.69 () {54 vars in 925 bytes} [Wed Aug 24 08:54:03 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:54:03.698999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100952-ef8d-4695-8a53-b59818a3e7fa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:03.699325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100952-ef8d-4695-8a53-b59818a3e7fa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:03.699609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100952-ef8d-4695-8a53-b59818a3e7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:03.759900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-61100952-ef8d-4695-8a53-b59818a3e7fa None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:03.761123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-61100952-ef8d-4695-8a53-b59818a3e7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:03.778390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:03.834402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 751/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 24 08:54:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:07.880641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f770-fdb8-4dcf-9fa5-d18fcdfb7591 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:07.881182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f770-fdb8-4dcf-9fa5-d18fcdfb7591 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:07.881321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f770-fdb8-4dcf-9fa5-d18fcdfb7591 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:07.882813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 768/1519] 10.222.0.64 () {54 vars in 961 bytes} [Wed Aug 24 08:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:07.887793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979297-5b64-499e-800d-b97d89574568 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:07.888246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979297-5b64-499e-800d-b97d89574568 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:07.888444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979297-5b64-499e-800d-b97d89574568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:07.907527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3b979297-5b64-499e-800d-b97d89574568 None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:07.908725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3b979297-5b64-499e-800d-b97d89574568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:07.917873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 752/1520] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Aug 24 08:54:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:07.925998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93f770-fdb8-4dcf-9fa5-d18fcdfb7591 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:07.939424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93f770-fdb8-4dcf-9fa5-d18fcdfb7591 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:07.944427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:07.945033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:07.945319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:07.945596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:07.953808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:07.955454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f658fe-1625-409d-a085-0e8f1961abf1 service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:07.965505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 769/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:14.487555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227ae74-7856-432c-ab5c-6107f121249f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:14.488127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227ae74-7856-432c-ab5c-6107f121249f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:14.488127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227ae74-7856-432c-ab5c-6107f121249f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:14.489472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:14.494362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268869f-c301-4256-b931-6fe8ce6ae73a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:14.494834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268869f-c301-4256-b931-6fe8ce6ae73a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:14.494934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268869f-c301-4256-b931-6fe8ce6ae73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:14.514173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8268869f-c301-4256-b931-6fe8ce6ae73a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:14.515422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8268869f-c301-4256-b931-6fe8ce6ae73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:14.524820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:14.677465 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227ae74-7856-432c-ab5c-6107f121249f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:14.692405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227ae74-7856-432c-ab5c-6107f121249f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:14.698217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:14.698848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:14.699127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:14.699408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:14.707832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:14.709291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963c555-b8f6-402e-acff-23fcd74e9dff service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:14.718892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 754/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:15.546107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ce830-3a82-4cb5-80c9-3f92daf00d99 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:15.546607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ce830-3a82-4cb5-80c9-3f92daf00d99 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:15.546759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ce830-3a82-4cb5-80c9-3f92daf00d99 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:15.548045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:15.552867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f43046-cb7b-4335-b515-a73eefabaec4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:15.553298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f43046-cb7b-4335-b515-a73eefabaec4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:15.553415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f43046-cb7b-4335-b515-a73eefabaec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:15.573981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-42f43046-cb7b-4335-b515-a73eefabaec4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:15.575255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-42f43046-cb7b-4335-b515-a73eefabaec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:15.583862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:15.736538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ce830-3a82-4cb5-80c9-3f92daf00d99 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:15.750553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ce830-3a82-4cb5-80c9-3f92daf00d99 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:15.755893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:15.756483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:15.756764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:15.757033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:15.764886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:15.766512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3182985-52e9-4c4c-9064-f26d8e18c5e3 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:15.776251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 772/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:19.922845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:19.923379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:19.923379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:19.924759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:19.930519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ca8d-5bdb-4682-8919-d697616f1361 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:19.930974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ca8d-5bdb-4682-8919-d697616f1361 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:19.931094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ca8d-5bdb-4682-8919-d697616f1361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:19.950096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a814ca8d-5bdb-4682-8919-d697616f1361 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:19.951326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a814ca8d-5bdb-4682-8919-d697616f1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:19.959718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:20.246830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:20.248549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:20.328683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8568be7-7d39-4fd8-bca6-e3f2ee272ea9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:20.332961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:20.333563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:20.333842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:20.334116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:20.343046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:20.344484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbc29c-7830-4dbb-bbae-789c4a06e62b service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:20.353754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 757/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:22.242422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2062-12e5-4ed4-8910-fc8b2be88a6a None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:22.242973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2062-12e5-4ed4-8910-fc8b2be88a6a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:22.242973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d2062-12e5-4ed4-8910-fc8b2be88a6a None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:22.244306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:22.249658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4e39b-d42a-4bf7-a7b7-d076550e349b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:22.250082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4e39b-d42a-4bf7-a7b7-d076550e349b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:22.250213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4e39b-d42a-4bf7-a7b7-d076550e349b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:22.270390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7de4e39b-d42a-4bf7-a7b7-d076550e349b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:22.271625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7de4e39b-d42a-4bf7-a7b7-d076550e349b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:22.279453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:22.572858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d2062-12e5-4ed4-8910-fc8b2be88a6a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:22.584196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d2062-12e5-4ed4-8910-fc8b2be88a6a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:22.587700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:22.588299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:22.588582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:22.588859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:22.598083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:22.599536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21471801-173f-4246-8367-b4e5253f8e4e service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:22.609260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 775/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:26.095173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:26.095786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:26.095786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:26.097643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:26.103012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756bfae-8864-4001-ac99-3dcc737f36a3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:26.103486 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756bfae-8864-4001-ac99-3dcc737f36a3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:26.103591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756bfae-8864-4001-ac99-3dcc737f36a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:26.123449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8756bfae-8864-4001-ac99-3dcc737f36a3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:26.124649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8756bfae-8864-4001-ac99-3dcc737f36a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:26.132441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:26.680900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:26.682605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:26.747539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:26.761108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfef36a-686a-46ea-a001-ab79221e6e1d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:26.764676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:26.765280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:26.765555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:26.765821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:26.774854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:26.776228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4088c-1ebe-42df-b7b8-803281a93057 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:26.785852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 760/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:28.716042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:28.716610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:28.716610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:28.717928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:54:28.723910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220790-0778-40c3-9001-995976ca8e82 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:28.724443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220790-0778-40c3-9001-995976ca8e82 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:28.724585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220790-0778-40c3-9001-995976ca8e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:28.744148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5d220790-0778-40c3-9001-995976ca8e82 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:54:28.745363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5d220790-0778-40c3-9001-995976ca8e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:28.753097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:54:29.316988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:54:29.318679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:54:29.332075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78225-9b2a-483a-b016-0347fcd16f68 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:54:29.335609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:54:29.336206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:54:29.336491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:54:29.336767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:54:29.345537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:54:29.347031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5cd91-6e88-4ade-a0c6-69253b5ee992 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:54:29.356223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 778/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:38.739476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=11 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:39.352307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:57.690434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.691569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.692181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.724442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:57.726913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:57.739680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 762/1540] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:57.749480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.750153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.750494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.793727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:57.795175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:57.807554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 779/1541] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:57.813917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.829381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83fbd-1c71-453b-8a3b-a5f1a1927522 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.839727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.840833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.841363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.841874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.842831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:57.843384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.845394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.867448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 763/1542] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:57.876407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.891756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e47c0-6ccc-48cd-9857-bb05df90b56b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.899432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.900157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.900739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.901025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.901877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.904185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d20ffb-09bb-4b87-90e5-74b95360442d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.055580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 780/1543] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:58.061093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.087179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de60d1f-cc85-49eb-8971-007756444708 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.157535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5081d-add1-4b2b-8eab-0d8c212c86a7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.157919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5081d-add1-4b2b-8eab-0d8c212c86a7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.158113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5081d-add1-4b2b-8eab-0d8c212c86a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.224736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-46d5081d-add1-4b2b-8eab-0d8c212c86a7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.226993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-46d5081d-add1-4b2b-8eab-0d8c212c86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.336103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.337120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.337630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.338122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.339549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.343406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.487512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 781/1544] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.500177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.500711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.501228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.505795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 764/1545] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/users => generated 291 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:58.511070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.560938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.566254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.582235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e096d4-8ec9-4e80-b2fc-5f5a98e901cf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.605193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.606184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.606712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.607195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.607788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 782/1546] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.611000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.611893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.615675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.646926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc29ee9-1d75-4d91-9a43-298f81939b55 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.664272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 765/1547] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:58.674950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.677047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.678180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.678691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.679205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.680427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:58.690303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.693458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:58.693458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:55:58.693458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:55:58.695289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9179-e5fb-4647-ab79-7d281038a111 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.709556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a449a-cb2d-488a-b7e4-c814cc679949 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.724285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.724924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.725225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.725505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.726174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.727980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.733860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 783/1548] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:58.737299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.737745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.738201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.759656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 766/1549] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:58.760450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.793099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060592-75b3-42ab-8ebd-5b16e45d9c3f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.801064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.804353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.805013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.805295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.805584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.806299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.807591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.814277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:58.814277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:55:58.814277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:55:58.815634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5993c1-ca1c-41a2-aecd-2067d18de714 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.844639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 784/1550] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.850700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.876238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73f573-967e-483e-a632-eea1f532a511 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.892030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.896684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.896684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.896684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] PATH_INFO: `/v3/projects/3fcabca864294bb79ed4dac5f846ae24/users/35a22d06334f4fc9b3a8556c430c433d/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.939865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcabca864294bb79ed4dac5f846ae24, user_id=35a22d06334f4fc9b3a8556c430c433d, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.943666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276fe5e-520a-44c1-9275-d18f77677f51 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.988248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 767/1551] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:58.990511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42ee0-ae4d-426b-bd41-06abe97b181b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.991180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42ee0-ae4d-426b-bd41-06abe97b181b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.992054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42ee0-ae4d-426b-bd41-06abe97b181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.026384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cbb42ee0-ae4d-426b-bd41-06abe97b181b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.028969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cbb42ee0-ae4d-426b-bd41-06abe97b181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.059125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 785/1552] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:58 2022] PUT /identity/v3/projects/3fcabca864294bb79ed4dac5f846ae24/users/35a22d06334f4fc9b3a8556c430c433d/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:59.061247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213173bb-f040-42f3-bfc5-d98e342f02e6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.061959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213173bb-f040-42f3-bfc5-d98e342f02e6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.062505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213173bb-f040-42f3-bfc5-d98e342f02e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.088363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-213173bb-f040-42f3-bfc5-d98e342f02e6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.090458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-213173bb-f040-42f3-bfc5-d98e342f02e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.174701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 768/1553] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.177378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.178151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.178596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.214400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 786/1554] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.214400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591566da-3c80-457f-a590-38f87a8469bb None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.214400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591566da-3c80-457f-a590-38f87a8469bb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.214400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591566da-3c80-457f-a590-38f87a8469bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.215108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.215437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.224692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 769/1555] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.231101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.240657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45561f1-e543-422e-b4b3-c170bdffc8cf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.243637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-591566da-3c80-457f-a590-38f87a8469bb None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.245046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-591566da-3c80-457f-a590-38f87a8469bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.247883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.248759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.249042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.249315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.250003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.253152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa096b01-881b-44c9-92e4-c1ee49358e2c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.254356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 787/1556] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.255782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8fd21-0c38-46f0-89d8-779f082e80ab None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.256100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8fd21-0c38-46f0-89d8-779f082e80ab None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.256414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8fd21-0c38-46f0-89d8-779f082e80ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.273116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.280128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-daf8fd21-0c38-46f0-89d8-779f082e80ab None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.281398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-daf8fd21-0c38-46f0-89d8-779f082e80ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.290542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 788/1557] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.293106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d5da-8814-4f6a-8286-387cbbe9c658 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.293337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d5da-8814-4f6a-8286-387cbbe9c658 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.293687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d5da-8814-4f6a-8286-387cbbe9c658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.330471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2b83d5da-8814-4f6a-8286-387cbbe9c658 None None] MFA Rules not processed for user `35a22d06334f4fc9b3a8556c430c433d`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.333434 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2b83d5da-8814-4f6a-8286-387cbbe9c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.382824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 770/1558] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:59.385207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.385701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.386192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.398594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.421019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.423096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.451539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 789/1559] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.451539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3e36e-2cbf-4fd7-9c2b-3b1fb23c08f6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.451539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3e36e-2cbf-4fd7-9c2b-3b1fb23c08f6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.451539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3e36e-2cbf-4fd7-9c2b-3b1fb23c08f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.452408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 771/1560] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.452408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.460265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9dc8-d90d-4ef7-8fc8-960cdf827494 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.467048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.467770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.468111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.468457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.469304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:59.469928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.470633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e8e3e36e-2cbf-4fd7-9c2b-3b1fb23c08f6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.471946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e8e3e36e-2cbf-4fd7-9c2b-3b1fb23c08f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.472300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8870157-125c-4be9-99c9-d8c7bbc1a9bb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.482501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 790/1561] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.485176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876900a-611a-4776-81a3-c83e00a52d53 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.485408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876900a-611a-4776-81a3-c83e00a52d53 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.485739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876900a-611a-4776-81a3-c83e00a52d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.493213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 772/1562] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:59.496142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.496557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.497026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.528503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c876900a-611a-4776-81a3-c83e00a52d53 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.530286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c876900a-611a-4776-81a3-c83e00a52d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.532571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.534711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.544771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 791/1563] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.546791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898849e1-553e-412e-b807-34df806c0b34 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.547263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898849e1-553e-412e-b807-34df806c0b34 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.547712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898849e1-553e-412e-b807-34df806c0b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.549498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 773/1564] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.555701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.569410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcd241-ed30-40c3-96ed-633e8051c52b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.575409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-898849e1-553e-412e-b807-34df806c0b34 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.576002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.576821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-898849e1-553e-412e-b807-34df806c0b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.577008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.577410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.577812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.578892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.581088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.587658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 792/1565] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.589204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898849e1-553e-412e-b807-34df806c0b34 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.600986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 774/1566] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:59.600986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.605708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898849e1-553e-412e-b807-34df806c0b34 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.611654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1e74c-76fe-4487-beda-de737c8f9d6b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.612866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.613742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.614226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.614706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.615733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.617966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.618221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.618849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.619186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.619614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.620355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:59.620707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.622521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74345a75-ee04-478b-b111-f9a94bd96282 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.641495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 775/1567] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:59.643625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.644095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.644568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.669052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.670793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.680681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 776/1568] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.682114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.696310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217ea3f-5b8c-4198-aefa-06eaedb2bab9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.702404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.703245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.703707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.704120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.705048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.707256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.730487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 793/1569] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:59.732219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.816787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24936968-8022-4f30-afa4-5565627a4198 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.823031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.823658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.823959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.824253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.824949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:59.825314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.827053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.845122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 794/1570] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:59.846088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 777/1571] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/projects => generated 403 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:59.848276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.855815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.870960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dc3a5-2076-4b77-a37d-6907044932e0 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.938625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.939036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d923baa-f525-4d8c-bed3-ca707380f443 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.947785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.948818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.957324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.957324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.957324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:59.957324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.957324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.960696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.961364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.961650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.961963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.962772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:59.963314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.965303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.987803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 778/1572] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:59.987803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.992450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 795/1573] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:59.997372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.000265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24365e-dac8-468e-bd22-a674f2c2ba60 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.006963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e029b6-6c61-4d81-ac43-f7b83f78ce0d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.009434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.010513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.011033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.011489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.012151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.012659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.012809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.013060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.013362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.014104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.016011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.016420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.036769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 779/1574] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:00.038412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.055142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd14a9-aca8-41f0-9277-d5bc34a2c543 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.062491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.063523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.063976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.064433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.065588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:00.066061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.069030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.090354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 780/1575] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:00.092037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.107380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022d475-7f21-4be6-8bb6-6f88755a5a68 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.112334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.112961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.113243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.113520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.114833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.116818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.158994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 796/1576] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.160118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.240093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c95981-7202-4c58-822c-4f04030a5e3a admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.245558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.246239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.246531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.246853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.247544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.248625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:56:00.248625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:56:00.248625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:56:00.249883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.257583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 781/1577] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.261124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.273099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99603954-b8fa-4c33-a6a6-63f634b360c4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.281057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.282088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.282536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.283037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.284175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.288384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.415617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 797/1578] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/users => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.416472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.428148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b9b2d-e938-4d01-98e7-95417dfe727b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.434440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.435415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.435804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.436182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.436953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.439374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.440984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 782/1579] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.442274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.470295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:00.533099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b402e-011c-4a44-b13d-5f4c695f9133 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.539097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.539733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.540018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.540436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.541245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.543564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.580889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 798/1580] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.581721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.661457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 783/1581] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.663104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.681819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0a9c9-4511-4243-b831-079f2beece8f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.723898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:00.758202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca486c-56b6-47e4-a01d-7e9711c41c7b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.764274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.765156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.765530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.766158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.803944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.804137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.804247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.804247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.804247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] PATH_INFO: `/v3/projects/dd27ac26660b46c388279e652200c7cd/users/2cd91f6976174fbf9692e0973435a4d5/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.804509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.804606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd27ac26660b46c388279e652200c7cd, user_id=2cd91f6976174fbf9692e0973435a4d5, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.804606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.906117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 784/1582] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:00.907067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.936630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 799/1583] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:00 2022] PUT /identity/v3/projects/dd27ac26660b46c388279e652200c7cd/users/2cd91f6976174fbf9692e0973435a4d5/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:00.938026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.993423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6be5-8248-40fe-9361-ba286f849cd7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.020767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939639e-706c-426b-9c5a-dcfdb5d7b62d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.026939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.027782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.028120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.028307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.029139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.031228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.033266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:01.044527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 800/1584] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.045526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.059088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c2ed4-946b-4dba-8d26-565def358e07 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.064165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.064801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.065083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.065358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.066006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.068272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.080120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 801/1585] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.081033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.082384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.083898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.084195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.084469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.085331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.087661 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.097433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b900d2-7541-4658-8568-58d1d8654f5c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.105611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.106599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.107091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.107538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.108586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.112123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.201941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 785/1586] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:01.203676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.214848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078866b-8c72-483a-b3f9-7e0a00bcac66 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.220248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.221238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.221606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.222019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.222836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.225056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acabb0-83fe-4e1c-9933-5a1a86f92954 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.273689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 802/1587] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:01.273689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.283639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f38ab-d263-4bb4-a584-8edabdb92dcc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.289534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.296922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.296922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.296922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.296922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.296922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.462313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 803/1588] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:01.462901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 786/1589] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:01.463327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.464654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.465155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.465557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.484159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d8c2d-6da3-4543-bd6c-52b2693189ed admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.492493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.493465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.493921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.494369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.495481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.497725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.500106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:01.501750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:01.513872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 787/1590] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:01.514965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.527597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c08cb8-f3e9-4c27-abf2-bcebaa7786e7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.533275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.534003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.534322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.534633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] PATH_INFO: `/v3/projects/42e036212c2f4448bd2cb35ede9ce408/users/bdd528e61e644a50a68748e0f4c8bc49/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.554172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e036212c2f4448bd2cb35ede9ce408, user_id=bdd528e61e644a50a68748e0f4c8bc49, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.555300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:56:01.555300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:56:01.555300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:56:01.557335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.603198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 804/1591] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:01.604735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.622524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442090-8dae-49c5-8091-baddd390fedf admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.630465 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 788/1592] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:01 2022] PUT /identity/v3/projects/42e036212c2f4448bd2cb35ede9ce408/users/bdd528e61e644a50a68748e0f4c8bc49/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:01.635212 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.696967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:01.719015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.719015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.719015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.719015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.719576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.721303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.723986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0341c-e2de-48c9-a294-ac57474c1171 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.729780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.730533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.731011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.731489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.732625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.734872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.743729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 805/1593] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.746187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.747865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 789/1594] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.748763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.760691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191634-493b-41aa-ae31-afe67fe6968f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.761101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1f5c-b304-4086-8a06-2a6d7208dab8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.765834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.766542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.766880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.767226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.767629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.767858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.768421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.768846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.769245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.769519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.770167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.772552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.781126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 790/1595] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.782110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.793904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbddd-c16c-4493-86a8-84ac71dfea8c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.794823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 806/1596] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.796154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.803344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.803982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.804264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.804546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.805206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.806910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.818810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 791/1597] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.819800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.830745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45390c2-3a96-467b-a1da-1d9807fdefcf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.835944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.836577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.836859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.837170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.837831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.839538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.851644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 792/1598] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.852514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.863315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c136e04-51b0-4a43-9dd3-989b3a1a08e4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.869223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.869871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.870149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.870423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.871316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.873033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.887818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 793/1599] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.887818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.895931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2852f-a630-4977-901a-82cdc0d876dc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.897015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c828534-0d1f-456a-b871-061101b8e617 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.901140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.901843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.902128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.902409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] PATH_INFO: `/v3/projects/97f62d7920c94505a55e187f1ea26f2c/users/79cdbd7e362b487f90fc13774fae686b/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.903490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.904570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.905079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.905577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.921561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.923151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f62d7920c94505a55e187f1ea26f2c, user_id=79cdbd7e362b487f90fc13774fae686b, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.924558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.928864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.940091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 807/1600] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:01.941117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.954421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805827b-d37b-4b6e-a3e6-e76eeb017c2f service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.963873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.964973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.965470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.965966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.967221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.970567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.991072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 808/1601] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:01.992613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.005638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f17a67-9f35-4e77-88fc-870ecf6f1165 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.010904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.011546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.011845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.012121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.012791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.014450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.026994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 809/1602] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:02.028069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.041969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a56698-3b0b-4371-8b5d-778daab5fbe9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.047295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.047938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.048215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.048493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.049155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.050903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.063178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 810/1603] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:02.067531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:02.067757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.072469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 794/1604] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:01 2022] PUT /identity/v3/projects/97f62d7920c94505a55e187f1ea26f2c/users/79cdbd7e362b487f90fc13774fae686b/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.074237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.081732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6492f9c-afa5-42f3-8485-5dd61b8f673a admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.166978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.167977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.168340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.168620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.169304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.171169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.186178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 811/1605] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:02.187423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.194619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407c5e-d266-4538-815e-f54061d974c4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.200895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69708f-7b51-4195-974a-896d55c07142 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.201589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.202632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.203171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.203923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.205311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.206208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.206927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.207212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.208011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] PATH_INFO: `/v3/projects/42e036212c2f4448bd2cb35ede9ce408/users/bdd528e61e644a50a68748e0f4c8bc49/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.208871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.210205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e036212c2f4448bd2cb35ede9ce408, user_id=bdd528e61e644a50a68748e0f4c8bc49, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.213262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.233643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 812/1606] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/42e036212c2f4448bd2cb35ede9ce408/users/bdd528e61e644a50a68748e0f4c8bc49/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.233643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.236487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 795/1607] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:02.238124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.254165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e01360-881a-47a4-983b-5165fb887230 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.263438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:02.322160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33100f1-52c3-44cc-8f6a-652fce5894c4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.328782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.329707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.331044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.331044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] PATH_INFO: `/v3/projects/574887b9d0fc42ce917c9b8b2c09e291/users/3f9025a5d28b47d8bc36f131e908382f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.331393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.331834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.332385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.333811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] PATH_INFO: `/v3/projects/6054f9b6abdf48cabd9ec6182c4dc955/users/77636e4bd66348f28b46d0b5f3b273f5/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.353704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574887b9d0fc42ce917c9b8b2c09e291, user_id=3f9025a5d28b47d8bc36f131e908382f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.359685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.362367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6054f9b6abdf48cabd9ec6182c4dc955, user_id=77636e4bd66348f28b46d0b5f3b273f5, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.367066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e737fc8-f3ed-4c24-a776-e0d3a72af5b5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.485828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 813/1608] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/574887b9d0fc42ce917c9b8b2c09e291/users/3f9025a5d28b47d8bc36f131e908382f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.486861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.487589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 796/1609] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/6054f9b6abdf48cabd9ec6182c4dc955/users/77636e4bd66348f28b46d0b5f3b273f5/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.489967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.490585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.491327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.546742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] MFA Rules not processed for user `79cdbd7e362b487f90fc13774fae686b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:02.548977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.569536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc17d6a-ff92-48c0-b18c-a7aa9661acd8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.658811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.659882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.660343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.660820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] PATH_INFO: `/v3/projects/c90f5f603c1f4edeac6252e231f247ae/users/e14b06fc52c049a197466bb83dd88f05/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.682423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90f5f603c1f4edeac6252e231f247ae, user_id=e14b06fc52c049a197466bb83dd88f05, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.685859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb9151-3853-4033-8a19-62d9973ee477 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.693622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 797/1610] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:02.695348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.723225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646618c3-c1db-4d91-a850-03439cbea3a5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.730077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.731775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.731950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.732442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] PATH_INFO: `/v3/projects/9ce953ed361742a29b0c750ca13d88e2/users/5171bf0514c3474aae358a9d7cfaf538/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.740051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:02.757242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce953ed361742a29b0c750ca13d88e2, user_id=5171bf0514c3474aae358a9d7cfaf538, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.760730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.770266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 814/1611] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/c90f5f603c1f4edeac6252e231f247ae/users/e14b06fc52c049a197466bb83dd88f05/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.771777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ee96-e2d5-4f20-bbb5-b196a7373cce None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.772579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ee96-e2d5-4f20-bbb5-b196a7373cce None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.772928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4ee96-e2d5-4f20-bbb5-b196a7373cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.803983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ccb4ee96-e2d5-4f20-bbb5-b196a7373cce None None] MFA Rules not processed for user `bdd528e61e644a50a68748e0f4c8bc49`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:02.805250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ccb4ee96-e2d5-4f20-bbb5-b196a7373cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.832897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:02.900415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 798/1612] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/9ce953ed361742a29b0c750ca13d88e2/users/5171bf0514c3474aae358a9d7cfaf538/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.902120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.918813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 815/1613] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:02.920939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153370a6-2b00-4864-ad54-d0d7e6460e94 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.921612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153370a6-2b00-4864-ad54-d0d7e6460e94 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.922221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153370a6-2b00-4864-ad54-d0d7e6460e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.947333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-153370a6-2b00-4864-ad54-d0d7e6460e94 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:02.948700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-153370a6-2b00-4864-ad54-d0d7e6460e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.022593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2601-8023-478d-b076-8c4166b9da36 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.032869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.039570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.039570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.039570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] PATH_INFO: `/v3/projects/f58ac192b94c47179055f35cae59f8ec/users/0737c1b1280c468fa9df3ba7bc9ee6b6/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.054434 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 816/1614] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.057217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3bb71-dcf0-43a3-a4b1-d635852f988f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.057981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3bb71-dcf0-43a3-a4b1-d635852f988f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.058726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3bb71-dcf0-43a3-a4b1-d635852f988f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.072552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58ac192b94c47179055f35cae59f8ec, user_id=0737c1b1280c468fa9df3ba7bc9ee6b6, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.079695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.109897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-68e3bb71-dcf0-43a3-a4b1-d635852f988f None None] MFA Rules not processed for user `77636e4bd66348f28b46d0b5f3b273f5`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.112054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-68e3bb71-dcf0-43a3-a4b1-d635852f988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.172960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 799/1615] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:02 2022] PUT /identity/v3/projects/f58ac192b94c47179055f35cae59f8ec/users/0737c1b1280c468fa9df3ba7bc9ee6b6/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:03.174501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.229005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 817/1616] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.234921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16920d52-7198-4ff1-b6fe-e6b49583202a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.235273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16920d52-7198-4ff1-b6fe-e6b49583202a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.235557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16920d52-7198-4ff1-b6fe-e6b49583202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.256405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-16920d52-7198-4ff1-b6fe-e6b49583202a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.263289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-16920d52-7198-4ff1-b6fe-e6b49583202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.266319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a0ef-afd8-48d8-851f-b474297a3361 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.276012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 818/1617] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.276012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.276012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.276012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.285250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.286702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.287065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.287389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.288060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.290513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.306857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.309407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.320933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 819/1618] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.322068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.376823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:03.394960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f506c3e-e1d3-4e9c-95c8-b21f342d8bb7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.395563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 800/1619] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/projects => generated 405 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:03.396485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.398320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.479425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.480317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.480466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.480650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.488758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf49b-0ed3-462c-947d-0869bda605fc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.494070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.495025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.495455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.495975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.503928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:03.567442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.577878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d47f5-9a0e-479c-b8eb-a5470e979fe6 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.580109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 820/1620] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.581414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.581700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.581975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.588345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.589870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989b0db-135f-44d2-bf35-e228ca4b0147 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.601766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 801/1621] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.603259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277ec7-6baa-4954-9978-05b7c53a7892 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.603549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277ec7-6baa-4954-9978-05b7c53a7892 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.603825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277ec7-6baa-4954-9978-05b7c53a7892 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.605676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 802/1622] 10.222.0.64 () {54 vars in 962 bytes} [Wed Aug 24 08:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:03.605833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.606478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277ec7-6baa-4954-9978-05b7c53a7892 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.607583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.618480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 821/1623] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.619630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277ec7-6baa-4954-9978-05b7c53a7892 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.619788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.626014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.626768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.627114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.627537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.631739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff5433-2182-40e9-b2b1-01dd193b2044 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.636575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.637351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.637631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.638005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.640283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.646736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.649109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.650602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513d7a-2e62-4218-a8e6-f9b4c2dbec3b service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.656057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 803/1624] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.658742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.662831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 822/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.664981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.665173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.665520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.683175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9034a4d-ee55-43ef-b9d6-e03066fb3b37 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.690489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.691223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.691526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.691996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.693283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.696520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.743245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.744572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.838131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 823/1626] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:03.843858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.848970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:03.859450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e17c7-d248-4647-b7ba-13302e268c9b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.859450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.859803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.860241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.860531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.871685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.873119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.884738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 824/1627] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.885531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.898933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e0f6-4f85-45f0-a6c9-497bdd8afd0c service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.916364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.916364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.916364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.916364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.956576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 804/1628] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/users => generated 309 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:03.956576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.986259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc818fe-2be3-4169-aed1-510ac7abd965 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.996725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.002232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.002232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.002232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.002232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.003777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.022349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 805/1629] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:04.023564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.025301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.028734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.038760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea07a8-ab9f-45ad-b67c-88bda1c69cf5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.043327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 825/1630] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:04.043535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.048342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.048342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.048342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.048836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.066031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5df-306b-4179-b326-ab465553e5e6 service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.076855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.078627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.079736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.080902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] PATH_INFO: `/v3/projects/8c68a07ef33646a2b1231693e2bae997/users/6919a663d83f41cf8b1c1151a3bf4d34/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.116203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c68a07ef33646a2b1231693e2bae997, user_id=6919a663d83f41cf8b1c1151a3bf4d34, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.122453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.130867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.132752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.157903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 806/1631] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:04.264144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 826/1632] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:04 2022] PUT /identity/v3/projects/8c68a07ef33646a2b1231693e2bae997/users/6919a663d83f41cf8b1c1151a3bf4d34/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:04.269131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.286267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1121-c060-4231-93d2-7359e02f111e service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.483731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.490872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.491311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.491802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.492863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.494701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c83d68-3078-45bc-8caf-27ef3a8fb4d9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.498690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:04.531853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 807/1633] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:04.534348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.542189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435713f8-4214-431c-afb6-7861753094dd None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.544862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435713f8-4214-431c-afb6-7861753094dd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.546393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435713f8-4214-431c-afb6-7861753094dd None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.556488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344c2cd-5427-4eed-9cc8-7f5935bdc135 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.557066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 808/1634] 10.222.0.64 () {54 vars in 912 bytes} [Wed Aug 24 08:56:04 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:04.567001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.567351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.568339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.575209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.580006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.580297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.582304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] PATH_INFO: `/v3/projects/8c68a07ef33646a2b1231693e2bae997/users/6919a663d83f41cf8b1c1151a3bf4d34/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.588390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c68a07ef33646a2b1231693e2bae997, user_id=6919a663d83f41cf8b1c1151a3bf4d34, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.591264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070d79e-2097-46f8-9f59-4cf25e365eb7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.622878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:04.625597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:04.628794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 827/1635] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:04 2022] PUT /identity/v3/projects/8c68a07ef33646a2b1231693e2bae997/users/6919a663d83f41cf8b1c1151a3bf4d34/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:04.648697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.649458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.649956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.713001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] MFA Rules not processed for user `6919a663d83f41cf8b1c1151a3bf4d34`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:04.715197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:04.884155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 809/1636] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:04.912872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.002701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56240233-4b14-4fa3-8086-2c4e6cbbe60f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.007158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.009423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.009423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.009423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.025751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.025751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.026815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 828/1637] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:05.038127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 810/1638] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:05.158691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.189329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68831046-348c-4142-b4e9-591dfc58f913 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.197593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.198096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.198382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.198880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.212023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.213658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac1685-f53c-498e-a718-caea1bd479c8 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.498297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 829/1639] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:05.556932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.683347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ce15-8749-454c-b0c1-b46bbd9cdab3 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.689382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.690391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.690857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.691275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] PATH_INFO: `/v3/users/79cdbd7e362b487f90fc13774fae686b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.694961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cdbd7e362b487f90fc13774fae686b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.696304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:56:05.696304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Aug 24 08:56:05.696304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: Aug 24 08:56:05.697567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.103126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:06.153990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cdbd7e362b487f90fc13774fae686b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:06.318480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cdbd7e362b487f90fc13774fae686b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:06.334390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cdbd7e362b487f90fc13774fae686b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:06.335367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cdbd7e362b487f90fc13774fae686b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:06.369588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cdbd7e362b487f90fc13774fae686b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:06.384308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b9eb-010d-4845-9735-24e33079a4a4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.385216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b9eb-010d-4845-9735-24e33079a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.385216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b9eb-010d-4845-9735-24e33079a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.388407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 811/1640] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:05 2022] DELETE /identity/v3/users/79cdbd7e362b487f90fc13774fae686b => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:06.420209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d5b6b9eb-010d-4845-9735-24e33079a4a4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:06.421840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d5b6b9eb-010d-4845-9735-24e33079a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:06.488479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:06.589748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 830/1641] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:06.657931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.659593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:06.813595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c7f4-8896-4ead-8eff-7e00344c9a49 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.817982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.819158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.819576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.820117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.835901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.841020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6cf63-06a3-430f-bafe-d3cb88bef195 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.861218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 812/1642] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:07.232740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fbc9d-cc3b-4669-a5c9-85e6d2b21727 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.233275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fbc9d-cc3b-4669-a5c9-85e6d2b21727 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.233785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fbc9d-cc3b-4669-a5c9-85e6d2b21727 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.236863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 831/1643] 10.222.0.64 () {54 vars in 962 bytes} [Wed Aug 24 08:56:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:07.252053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706af68-b685-4b6d-8402-36bd58baebcc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.252707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706af68-b685-4b6d-8402-36bd58baebcc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.253026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706af68-b685-4b6d-8402-36bd58baebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.330475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5706af68-b685-4b6d-8402-36bd58baebcc None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:07.334585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5706af68-b685-4b6d-8402-36bd58baebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:07.525827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 813/1644] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Aug 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:07.531131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.541580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.541580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.550564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 832/1645] 10.222.0.64 () {54 vars in 912 bytes} [Wed Aug 24 08:56:07 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:07.572529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.573331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.574106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.616731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:07.618964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:07.676616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 814/1646] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:07.694544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:07.696822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:07.825815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096928-3166-4db2-9218-fd749c77f290 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.831060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.836792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.836792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.836792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.848041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.849995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ce33f-7234-4d42-84c4-1fb864f6cdd8 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:07.886602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 833/1647] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:07.907592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:07.909986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:07.953322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b7650-7d0e-430f-910a-cada46b11e2a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.959286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.960233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.961236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.961572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] PATH_INFO: `/v3/projects/97f62d7920c94505a55e187f1ea26f2c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.978024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f62d7920c94505a55e187f1ea26f2c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.987379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.136014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.137720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.137720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.215549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] MFA Rules not processed for user `e14b06fc52c049a197466bb83dd88f05`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:08.217307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:08.220344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.275391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f62d7920c94505a55e187f1ea26f2c', 'request_id': 'req-73cdf682-369e-4350-bd77-e6baf9e774a6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.311681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f62d7920c94505a55e187f1ea26f2c', 'request_id': 'req-73cdf682-369e-4350-bd77-e6baf9e774a6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.415464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f62d7920c94505a55e187f1ea26f2c', 'request_id': 'req-73cdf682-369e-4350-bd77-e6baf9e774a6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.441447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] The token cache is being invalidate because project 97f62d7920c94505a55e187f1ea26f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:08.442693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.446039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.451724 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 815/1648] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:07 2022] DELETE /identity/v3/projects/97f62d7920c94505a55e187f1ea26f2c => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:08.460788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.463723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.533239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 834/1649] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:08.552080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.562386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.758853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdf682-369e-4350-bd77-e6baf9e774a6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.779512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.780989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.781129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.783917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] PATH_INFO: `/v3/users/6919a663d83f41cf8b1c1151a3bf4d34` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.785348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:08.785497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6919a663d83f41cf8b1c1151a3bf4d34)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.790838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.805971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7376-fed8-4b1e-9c91-b696ec727b9c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.813805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.814563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.815027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.815407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.816395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.819313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.975279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 835/1650] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:08 2022] POST /identity/v3/projects => generated 373 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:08.993624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.997641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.139566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6919a663d83f41cf8b1c1151a3bf4d34'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:09.220701 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6919a663d83f41cf8b1c1151a3bf4d34'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:09.234598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6919a663d83f41cf8b1c1151a3bf4d34'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:09.235644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6919a663d83f41cf8b1c1151a3bf4d34'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:09.267978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6919a663d83f41cf8b1c1151a3bf4d34'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:09.283509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 816/1651] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:08 2022] DELETE /identity/v3/users/6919a663d83f41cf8b1c1151a3bf4d34 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:09.330837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27708-ca3c-4cd2-9c87-d1848162672b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.489255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.489909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.490235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.493117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.494904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.503243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.667103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 836/1652] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:08 2022] POST /identity/v3/users => generated 293 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:09.668628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:09.670134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.712943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d488b-f72c-4bd6-83c3-28fb545b79fb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.735130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.743303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.744762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.747784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.747784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.757744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.799712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 817/1653] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:09.800728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:09.825417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c496b-20c0-4e11-9b90-f62ff3ec6efd admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.845823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.847273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.848616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.849864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] PATH_INFO: `/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9/users/4d1b5ef6c30249b28d1df8e7ded7a5e1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.881805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e8fbf8752b418cbe4204f4b95f5ec9, user_id=4d1b5ef6c30249b28d1df8e7ded7a5e1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.913260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.060864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 837/1654] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:09 2022] PUT /identity/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9/users/4d1b5ef6c30249b28d1df8e7ded7a5e1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:10.064981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.078262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94ea3b-ec01-4475-9f2c-a65c1d6da663 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.116472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:10.171177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.172148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.172714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.173011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.173886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.176885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.215762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 818/1655] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:10.220741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.263748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76682e-80d8-4756-bab2-3fb706926269 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.286760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.288897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.289325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.290013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] PATH_INFO: `/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9/users/4d1b5ef6c30249b28d1df8e7ded7a5e1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.296804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e8fbf8752b418cbe4204f4b95f5ec9, user_id=4d1b5ef6c30249b28d1df8e7ded7a5e1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.307830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e2ea2-6878-4c16-a0d2-1de4276d0c51 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.352969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 838/1656] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:10 2022] PUT /identity/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9/users/4d1b5ef6c30249b28d1df8e7ded7a5e1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:10.735321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.740519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:10.783362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89e64e-50d6-404c-9df0-478662bf1d67 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.931029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.931462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.931755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.932089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] PATH_INFO: `/v3/projects/8c68a07ef33646a2b1231693e2bae997` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.933700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c68a07ef33646a2b1231693e2bae997)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.935973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.076032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c68a07ef33646a2b1231693e2bae997', 'request_id': 'req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.086594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c68a07ef33646a2b1231693e2bae997', 'request_id': 'req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.204194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c68a07ef33646a2b1231693e2bae997', 'request_id': 'req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.225450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] The token cache is being invalidate because project 8c68a07ef33646a2b1231693e2bae997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:11.226452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.227232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5ea8fa65-f3ed-4266-b462-0c3d6dce36f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.229231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 819/1657] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:10 2022] DELETE /identity/v3/projects/8c68a07ef33646a2b1231693e2bae997 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:11.236393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.237082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.237082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.296870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:11.299792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.405936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:11.488432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 839/1658] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:11.490807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.491091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.491636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.587853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:11.592860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.620554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.633850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 820/1659] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:11.645820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeed72f-008b-474d-b130-467dbf1fbe97 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:11.665566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:11.666054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.666344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.666616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.667349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:11.668586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.669708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.685938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 840/1660] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:11.696808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.718491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3b60-2f39-4149-8a60-4478ac67d070 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:11.730862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:11.732101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.732613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.734347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.735657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.739697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.894950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 821/1661] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:11.896411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.915805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe9e6a-af7b-45f2-bbdf-db0772dd74ca admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.061009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.061803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.062202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.062725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.064158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.071397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.289389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 841/1662] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/users => generated 283 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:12.296507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:12.323560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e804d-ae94-406c-adc7-bb9ed2cf9d65 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.336208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.337547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.338057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.338592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.339568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.342781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.362155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 822/1663] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:12.363133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:12.380429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa84fc-67d3-4111-924d-4082e603fef7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.390431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.392449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.393048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.393748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.395693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.403735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25898ca4-0dee-474a-86d1-e038198cc7c2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.438814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:12.446681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 842/1664] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:12.451102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07373-20ce-4b7a-8f00-e7074644295a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.460872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.461954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.462435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.463785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] PATH_INFO: `/v3/projects/d22b264a068b4563b9a70fe660d2ad03/users/7a098cc970514a76b86b03ca4c8a23ec/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.495043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22b264a068b4563b9a70fe660d2ad03, user_id=7a098cc970514a76b86b03ca4c8a23ec, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.501320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.538043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:12.646603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 823/1665] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:12 2022] PUT /identity/v3/projects/d22b264a068b4563b9a70fe660d2ad03/users/7a098cc970514a76b86b03ca4c8a23ec/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:12.658137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.658137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.658137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.703052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] MFA Rules not processed for user `7a098cc970514a76b86b03ca4c8a23ec`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:12.706727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:12.908083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 843/1666] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:12.909766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.104391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c836f-a5de-47aa-bb8d-38582e0104fc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.117899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.118581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.118895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.121903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.122892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.135433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.311898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.322890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 824/1667] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:12 2022] POST /identity/v3/projects => generated 363 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:13.328960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41466-86ef-493f-b94f-1345adb541a6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.437217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:13.472810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.474358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.474807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.478873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.478873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.478873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2c85-0ceb-4cd4-a9a8-b13d1fc3f05e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.646610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.654029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 844/1668] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:13 2022] POST /identity/v3/users => generated 288 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:13.666007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca46a6-f12d-42e4-a190-8010f35e629d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.676406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.677424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.677714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.678761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.679673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.698605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.703096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.703818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.704331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.748912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 825/1669] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:13.759198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.802478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa47d-8550-4756-8961-90f1476027a4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.811114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] MFA Rules not processed for user `4d1b5ef6c30249b28d1df8e7ded7a5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:13.816519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:13.820141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.821306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.821306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.821474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] PATH_INFO: `/v3/projects/9625eeb117a14241b382539f85004f67/users/a4c298a39a9c48d6bc09d94bb3704516/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.900239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9625eeb117a14241b382539f85004f67, user_id=a4c298a39a9c48d6bc09d94bb3704516, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.908194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.045706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 826/1670] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:13 2022] PUT /identity/v3/projects/9625eeb117a14241b382539f85004f67/users/a4c298a39a9c48d6bc09d94bb3704516/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:14.049656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.057449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 845/1671] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:14.069207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.073231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.253543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e41fa-add5-4c0c-b777-4477f0977c41 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:14.281796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3929df-a902-4da0-9ebe-749388d42d5b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:14.292126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:14.292764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.293489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.293574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.294277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:14.295975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.351430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 827/1672] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:14.351430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.373745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588d61e-5c5e-4dac-8997-69684d9e34ae admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:14.389078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:14.389778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.391076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.391573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] PATH_INFO: `/v3/projects/9625eeb117a14241b382539f85004f67/users/a4c298a39a9c48d6bc09d94bb3704516/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.394566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9625eeb117a14241b382539f85004f67, user_id=a4c298a39a9c48d6bc09d94bb3704516, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:14.399782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2143a5-a1c9-4ddb-9884-5e7f19b93dab admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.422428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 828/1673] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:14 2022] PUT /identity/v3/projects/9625eeb117a14241b382539f85004f67/users/a4c298a39a9c48d6bc09d94bb3704516/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:14.426724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:14.427333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.427799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.428230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.430575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.437296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.438645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.439042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.490225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] MFA Rules not processed for user `a4c298a39a9c48d6bc09d94bb3704516`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:14.491683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.651531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:14.653454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.675289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 829/1674] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:14.689534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:14.702416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.709432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.713624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 846/1675] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:14.904392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:14.978442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62f54d-f081-4d3e-948d-da28784b6e6d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:15.001586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:15.002320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:15.002546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:15.002912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:15.056877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:15.061165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b127a1f-16b6-4f03-bc00-5cf4beea50c5 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:15.112884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 830/1676] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:17.203586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:17.225929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131c681-69ca-4468-8116-bfcfd61dd535 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:17.456971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:17.457644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.457978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.458400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] PATH_INFO: `/v3/users/7a098cc970514a76b86b03ca4c8a23ec` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.460057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a098cc970514a76b86b03ca4c8a23ec)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.464207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.865685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a098cc970514a76b86b03ca4c8a23ec'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:17.866464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a098cc970514a76b86b03ca4c8a23ec'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:17.959621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a098cc970514a76b86b03ca4c8a23ec'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:17.997022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a098cc970514a76b86b03ca4c8a23ec'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:18.061411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a098cc970514a76b86b03ca4c8a23ec'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:18.090828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 847/1677] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:17 2022] DELETE /identity/v3/users/7a098cc970514a76b86b03ca4c8a23ec => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:18.096475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:18.097075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:18.097377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:18.160757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:18.164508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:18.520598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 831/1678] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:18.604317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:18.608686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:18.853074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:18.892947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16940601-d254-43a3-891d-c1804ff44da8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:18.902543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:18.903489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:18.903933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:18.904546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:18.912934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:19.058698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.062827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:19.068753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.083645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee7e0-aa2d-4c68-a3cb-d74a7cd20383 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.096707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 848/1679] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:19.425747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4420d7d-8c96-45bf-b0a0-ca1caec9bed2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:19.438115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:19.440089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.440089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.440423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.499073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:19.511718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.516972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.558310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 832/1680] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:19.732426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c70a-2887-420a-823e-bc4153b8e3e4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.733044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c70a-2887-420a-823e-bc4153b8e3e4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.734388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c70a-2887-420a-823e-bc4153b8e3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.815113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1e93c70a-2887-420a-823e-bc4153b8e3e4 None None] MFA Rules not processed for user `3f9025a5d28b47d8bc36f131e908382f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:19.818082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1e93c70a-2887-420a-823e-bc4153b8e3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:19.978312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 849/1681] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:20.006148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:20.009414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.217943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727902-5ba5-41ea-8c28-5eb444b54a2f service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.239219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.242552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.243896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.244144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.244740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.245103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.245454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.294589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.300568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.334491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 833/1682] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:20.401815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] MFA Rules not processed for user `2cd91f6976174fbf9692e0973435a4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:20.410199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.643132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 850/1683] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:20.669909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:20.672101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.696016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88debdd-0254-46da-aa4a-8586d6c12c24 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.706480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.707367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.716705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.717062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.738882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.741259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.755261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 834/1684] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:21.277293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.279450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.305032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21926f45-6bcf-4400-a0dc-0081fa7d3fab None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.322765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.323695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.324127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.324571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] PATH_INFO: `/v3/projects/d22b264a068b4563b9a70fe660d2ad03` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.326055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22b264a068b4563b9a70fe660d2ad03)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.330547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: WARNING py.warnings [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:56:21.330547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: warnings.warn(msg) Aug 24 08:56:21.330547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: Aug 24 08:56:21.330547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.335121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:21.579206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22b264a068b4563b9a70fe660d2ad03', 'request_id': 'req-3277e956-47a7-4e6b-9215-1606d30d7693'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.594383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22b264a068b4563b9a70fe660d2ad03', 'request_id': 'req-3277e956-47a7-4e6b-9215-1606d30d7693'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.692183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22b264a068b4563b9a70fe660d2ad03', 'request_id': 'req-3277e956-47a7-4e6b-9215-1606d30d7693'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.789915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] The token cache is being invalidate because project d22b264a068b4563b9a70fe660d2ad03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:21.790933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.802785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.809693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 851/1685] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/projects/d22b264a068b4563b9a70fe660d2ad03 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:21.810268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.833416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88dab6-ae06-4ff4-971a-66419e0d7df5 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.900622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.904714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.070386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.071410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.071898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.072903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] PATH_INFO: `/v3/users/a4c298a39a9c48d6bc09d94bb3704516` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.080023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c298a39a9c48d6bc09d94bb3704516)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.082878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.187725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277e956-47a7-4e6b-9215-1606d30d7693 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.201416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.202318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.202775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.203340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] PATH_INFO: `/v3/users/77636e4bd66348f28b46d0b5f3b273f5` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.205852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=77636e4bd66348f28b46d0b5f3b273f5)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.220949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.328679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c298a39a9c48d6bc09d94bb3704516'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.475740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c298a39a9c48d6bc09d94bb3704516'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.498784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c298a39a9c48d6bc09d94bb3704516'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.501179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c298a39a9c48d6bc09d94bb3704516'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.518933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77636e4bd66348f28b46d0b5f3b273f5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.521718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:22.525351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77636e4bd66348f28b46d0b5f3b273f5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.568333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c298a39a9c48d6bc09d94bb3704516'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.568333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 835/1686] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/users/a4c298a39a9c48d6bc09d94bb3704516 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:22.568333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.568333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.723206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951410-2744-41bd-897d-fb51c97f8ae3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.730588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.731752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.732182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.732697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] PATH_INFO: `/v3/users/35a22d06334f4fc9b3a8556c430c433d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.734281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a22d06334f4fc9b3a8556c430c433d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.736628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.813814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77636e4bd66348f28b46d0b5f3b273f5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.815866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77636e4bd66348f28b46d0b5f3b273f5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.849912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-26d9a33b-3369-4db1-b8d4-d0df40152cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77636e4bd66348f28b46d0b5f3b273f5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.859352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 852/1687] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/users/77636e4bd66348f28b46d0b5f3b273f5 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:22.860912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec3b7-0dc6-4131-bb39-3bd8c830dc31 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.869798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec3b7-0dc6-4131-bb39-3bd8c830dc31 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.876652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec3b7-0dc6-4131-bb39-3bd8c830dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.906819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:22.922982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8c5ec3b7-0dc6-4131-bb39-3bd8c830dc31 None None] MFA Rules not processed for user `5171bf0514c3474aae358a9d7cfaf538`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:22.924250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8c5ec3b7-0dc6-4131-bb39-3bd8c830dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.119531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 853/1688] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:23.119531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113fe64-7923-4ee3-90ec-77d041016710 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.119531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113fe64-7923-4ee3-90ec-77d041016710 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.119531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113fe64-7923-4ee3-90ec-77d041016710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.144735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a113fe64-7923-4ee3-90ec-77d041016710 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:23.146746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a113fe64-7923-4ee3-90ec-77d041016710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.216511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a22d06334f4fc9b3a8556c430c433d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:23.304116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 854/1689] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:23.306055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.307548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.309098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.361833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] MFA Rules not processed for user `5171bf0514c3474aae358a9d7cfaf538`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:23.363222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.367030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a22d06334f4fc9b3a8556c430c433d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:23.378084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a22d06334f4fc9b3a8556c430c433d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:23.379051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a22d06334f4fc9b3a8556c430c433d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:23.404877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 855/1690] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:23.407687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:23.408668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e0391a2-0c6d-477a-a17c-d7f02bf3d249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a22d06334f4fc9b3a8556c430c433d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:23.411570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.421156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 836/1691] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:22 2022] DELETE /identity/v3/users/35a22d06334f4fc9b3a8556c430c433d => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:23.425367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.425945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.426230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.502528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:23.511997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.576623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a351-ff15-4f88-badf-661a0bb9c6c3 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:23.780120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:23.781974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.782555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.785012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.790565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.825771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 837/1692] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:23.831838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:23.834281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:23.840051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:23.851713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.865089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 856/1693] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:23.963967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:23.983716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab893e3-d852-40b3-946e-2cc033c85329 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:23.988615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:23.996973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.996973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.996973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.006883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:24.008980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:24.023726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 857/1694] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:24.080691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f711dcb-ce2b-4de5-8000-0ed9e096e5e3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:24.088498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:24.089134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.089412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.089927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.092774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:24.235330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:24.237540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268e18d-b5cd-4ca9-a5f1-82785fd01be5 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:24.260361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 838/1695] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:24.352766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:24.356022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:24.374455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413afed-417e-4b2a-b1ad-89f32a2af60c service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:24.409182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:24.410038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.410328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.410606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] PATH_INFO: `/v3/projects/9625eeb117a14241b382539f85004f67` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.416714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9625eeb117a14241b382539f85004f67)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:24.419638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:24.613104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9625eeb117a14241b382539f85004f67', 'request_id': 'req-4f59696a-ad65-4ff9-b424-2a32947274a1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.637016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9625eeb117a14241b382539f85004f67', 'request_id': 'req-4f59696a-ad65-4ff9-b424-2a32947274a1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.714999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc5a78-5109-4a66-b489-702f1aedd4bc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.718630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc5a78-5109-4a66-b489-702f1aedd4bc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.718630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc5a78-5109-4a66-b489-702f1aedd4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.739785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9625eeb117a14241b382539f85004f67', 'request_id': 'req-4f59696a-ad65-4ff9-b424-2a32947274a1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.782753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] The token cache is being invalidate because project 9625eeb117a14241b382539f85004f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:24.786683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.789687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4f59696a-ad65-4ff9-b424-2a32947274a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.794915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 858/1696] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:24 2022] DELETE /identity/v3/projects/9625eeb117a14241b382539f85004f67 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:24.802694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b33a4-9559-459b-a54b-1943f86d9394 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.806282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b33a4-9559-459b-a54b-1943f86d9394 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.806839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b33a4-9559-459b-a54b-1943f86d9394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.818916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-84fc5a78-5109-4a66-b489-702f1aedd4bc None None] MFA Rules not processed for user `0737c1b1280c468fa9df3ba7bc9ee6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:24.820403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-84fc5a78-5109-4a66-b489-702f1aedd4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:24.874042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f44b33a4-9559-459b-a54b-1943f86d9394 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:24.875758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f44b33a4-9559-459b-a54b-1943f86d9394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:24.995456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 839/1697] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:25.009745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.010250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.010739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.042920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] MFA Rules not processed for user `0737c1b1280c468fa9df3ba7bc9ee6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:25.045226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.059317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 859/1698] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:25.071407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.071741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.072027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.072681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 840/1699] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:25.090190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.092791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.127875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:25.132731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.159467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 860/1700] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:25.165169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.201610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb13a1-c16f-4100-89ab-271ac647df20 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.214003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.214707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.216733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.216891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.218295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:25.219534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.222203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.253027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 861/1701] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:25.254291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.277483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a928bde-7d44-417f-adab-442d535cdf57 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.295326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.297082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.298225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.298818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.299953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.303727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.322477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf79ef8-8c83-428a-9120-1877c4e442ee None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.330688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.331664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.332214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.332709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.345258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.349153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.369729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 841/1702] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:25.370936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.373151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.392487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998355c-7565-4bd2-acad-a3ac92d2d3df service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.400214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.402082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.402082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.402082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] PATH_INFO: `/v3/projects/3fcabca864294bb79ed4dac5f846ae24` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.403359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcabca864294bb79ed4dac5f846ae24)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.406463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.472663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 862/1703] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:25.478431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.605965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcabca864294bb79ed4dac5f846ae24', 'request_id': 'req-f8cfd813-27a1-4af3-9cce-6eed48d416c5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.612706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcabca864294bb79ed4dac5f846ae24', 'request_id': 'req-f8cfd813-27a1-4af3-9cce-6eed48d416c5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.689369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcabca864294bb79ed4dac5f846ae24', 'request_id': 'req-f8cfd813-27a1-4af3-9cce-6eed48d416c5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.696321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b802c7-93bc-45e7-b6b1-52a6c8e51dca admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.709418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] The token cache is being invalidate because project 3fcabca864294bb79ed4dac5f846ae24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:25.710000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.711708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.716436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 842/1704] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:25 2022] DELETE /identity/v3/projects/3fcabca864294bb79ed4dac5f846ae24 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:25.720280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.722180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.844626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.846784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.847077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.847432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.848121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.857744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03a661-ce88-44ad-9923-a4d3cc636754 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.874135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfd813-27a1-4af3-9cce-6eed48d416c5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.886598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.887582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.887995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.888379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] PATH_INFO: `/v3/users/bdd528e61e644a50a68748e0f4c8bc49` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.891817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd528e61e644a50a68748e0f4c8bc49)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.894056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.901929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:26.010698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 863/1705] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/users => generated 293 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:26.010698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.010698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.010698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.010698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 864/1706] 10.222.0.64 () {54 vars in 912 bytes} [Wed Aug 24 08:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:26.012324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.025835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.070984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7c12-2426-429a-9144-3b18fca8319f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.082133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.083501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.083658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.083945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] PATH_INFO: `/v3/projects/6054f9b6abdf48cabd9ec6182c4dc955` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.086843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6054f9b6abdf48cabd9ec6182c4dc955)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.089225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.094975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:26.183698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd528e61e644a50a68748e0f4c8bc49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.273359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd528e61e644a50a68748e0f4c8bc49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.313222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd528e61e644a50a68748e0f4c8bc49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.316042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd528e61e644a50a68748e0f4c8bc49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.387866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6054f9b6abdf48cabd9ec6182c4dc955', 'request_id': 'req-326cad23-4845-49cb-95e4-c1a123e45b22'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.404164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd528e61e644a50a68748e0f4c8bc49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.418442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6054f9b6abdf48cabd9ec6182c4dc955', 'request_id': 'req-326cad23-4845-49cb-95e4-c1a123e45b22'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.422142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 843/1707] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:25 2022] DELETE /identity/v3/users/bdd528e61e644a50a68748e0f4c8bc49 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:26.424632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.428266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.522013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0bac9-4231-49b6-8de4-0a53559f260c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.531806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6054f9b6abdf48cabd9ec6182c4dc955', 'request_id': 'req-326cad23-4845-49cb-95e4-c1a123e45b22'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.551101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.553548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] The token cache is being invalidate because project 6054f9b6abdf48cabd9ec6182c4dc955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:26.554430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.554987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.555491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.556656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-326cad23-4845-49cb-95e4-c1a123e45b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:26.556984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.560176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.562253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a54de-25e0-48f1-a304-ba9972a16a43 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.562947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 865/1708] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:26 2022] DELETE /identity/v3/projects/6054f9b6abdf48cabd9ec6182c4dc955 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:26.565866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2c8e2-51eb-4826-84db-e3262264c167 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.566397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2c8e2-51eb-4826-84db-e3262264c167 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.566946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2c8e2-51eb-4826-84db-e3262264c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.591239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 844/1709] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.594753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5287d-9722-4d7b-a087-18931dc799c0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.595441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5287d-9722-4d7b-a087-18931dc799c0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.596158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5287d-9722-4d7b-a087-18931dc799c0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.604732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 845/1710] 10.222.0.64 () {54 vars in 912 bytes} [Wed Aug 24 08:56:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:26.609975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.611931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.612459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.657960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.660837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.668090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c1a2c8e2-51eb-4826-84db-e3262264c167 None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.680640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 846/1711] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:26.681232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c1a2c8e2-51eb-4826-84db-e3262264c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.681379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.681379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.689853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583008e-89d6-4749-ab16-bfc199a1977d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.705009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.706405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.706939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.707437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.708433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.710614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a4d7b-b3d0-447a-9a30-af8be350ff11 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.738016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 847/1712] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.738016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b1d7-f7d0-407c-83f1-23303b28f8d5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.738016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b1d7-f7d0-407c-83f1-23303b28f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.738016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b1d7-f7d0-407c-83f1-23303b28f8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.776484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5be2b1d7-f7d0-407c-83f1-23303b28f8d5 None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.778482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5be2b1d7-f7d0-407c-83f1-23303b28f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.866870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 866/1713] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:26.868922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.869363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.869810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.889383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 848/1714] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:26.891202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.891600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.891982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.912338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.917596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.934581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] MFA Rules not processed for user `3f9025a5d28b47d8bc36f131e908382f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.937827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 867/1715] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:26.938948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.939586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.964327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93186-6829-46fb-9dc9-314e26db6a2d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.971471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.972489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.972945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.973600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] PATH_INFO: `/v3/projects/065be83bf9f44e6ea20d7bddb44ac3f3/users/4ad2ba3c966f44848a78b8185e61a505/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.998877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065be83bf9f44e6ea20d7bddb44ac3f3, user_id=4ad2ba3c966f44848a78b8185e61a505, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.000550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.070585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 849/1716] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:27.082939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.084489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:27.101316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:27.113275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5720a5-f64f-4aaf-901f-9859aaab0012 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.117956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.118405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.119245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.119245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.142313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.142313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.148010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 868/1717] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:26 2022] PUT /identity/v3/projects/065be83bf9f44e6ea20d7bddb44ac3f3/users/4ad2ba3c966f44848a78b8185e61a505/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:27.149250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.161984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 850/1718] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:27.163040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.164514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:27.186272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f739b-05fa-4dc7-9597-9c0030826aa9 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.273063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322cbbd-8e9e-4148-99d3-55dc98a94ce3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.279840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.280984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.281473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.282254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.283307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:27.283801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.285844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624b9ef-8118-4b71-940b-51f541da16e0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.291812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.292680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.293133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.293600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.305887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 869/1719] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:27.305887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.305887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.305887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.315410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.334314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.335945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 851/1720] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:27.337291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.349840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] MFA Rules not processed for user `4ad2ba3c966f44848a78b8185e61a505`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:27.351020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577c419-0a44-4b88-82a1-b25dc1b5dec1 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.352240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:27.357032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.364116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.365332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.365823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.381394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.384153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.462940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 870/1721] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:27.467491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.481990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a56669-b1b7-42f9-be4a-e5e6e8cd6cf9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.487881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.490805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.491186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.491539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.492325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.517852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.641315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 852/1722] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/projects => generated 373 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:27.652376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.707538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 871/1723] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:27.710835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.935248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d791e-6e43-4632-b32c-15de328810dd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.938199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781ce0d-66f2-411c-ae48-4800de01f066 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.946545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.948469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.949991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.952820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.954915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.955452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.955564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.955829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.956143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.957290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.959412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.982702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.096528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 872/1724] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/users => generated 294 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:28.097527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.101775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.128123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 853/1725] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/users => generated 287 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:28.129056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.146290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052871d0-adea-4062-b03c-1a2443bff956 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.148854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f690de2-e858-4f4e-8dc0-1e6b7a2c459c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.153691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.155946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.156290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.156577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] PATH_INFO: `/v3/projects/42e036212c2f4448bd2cb35ede9ce408` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.157982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e036212c2f4448bd2cb35ede9ce408)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.160324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.163059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.164031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.164396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.164809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.165637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.170219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.208892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 854/1726] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:28.210694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.240904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139bb7-442b-4e7d-805a-b8f7130247e2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.248692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.249492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.249492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.249935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.251695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.253880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.271770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 855/1727] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:28.272747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.312155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cf50e-06d4-4bdf-b94f-1247e8e7555a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.328485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.329531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.329826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.330179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.331197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.335329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.371331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:28.391679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 856/1728] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:28.391888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e036212c2f4448bd2cb35ede9ce408', 'request_id': 'req-b3e6ed12-8a6c-406b-bc29-aace5067b641'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:28.392533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.405338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e036212c2f4448bd2cb35ede9ce408', 'request_id': 'req-b3e6ed12-8a6c-406b-bc29-aace5067b641'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:28.432751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb6419-9240-4e79-b007-e9914e706102 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.556351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e036212c2f4448bd2cb35ede9ce408', 'request_id': 'req-b3e6ed12-8a6c-406b-bc29-aace5067b641'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:28.580306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] The token cache is being invalidate because project 42e036212c2f4448bd2cb35ede9ce408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:28.580876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:28.583696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:28.585798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 873/1729] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:28 2022] DELETE /identity/v3/projects/42e036212c2f4448bd2cb35ede9ce408 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:28.587571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.589892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.612378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.613023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.613332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.613572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.614263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.616086 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10b697-506d-4b77-ace8-471a28d9b33c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.675496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 857/1730] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:28.676165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2176c2-7b7f-4cb1-bf14-69b68b24acba None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.676406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2176c2-7b7f-4cb1-bf14-69b68b24acba None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.676691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2176c2-7b7f-4cb1-bf14-69b68b24acba None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.681698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 858/1731] 10.222.0.64 () {54 vars in 961 bytes} [Wed Aug 24 08:56:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:28.685222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718596-ee61-461f-adc5-6aa7a4cdc33e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.685511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718596-ee61-461f-adc5-6aa7a4cdc33e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.685935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718596-ee61-461f-adc5-6aa7a4cdc33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.736171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ed12-8a6c-406b-bc29-aace5067b641 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.743140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.744266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.744656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.745770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] PATH_INFO: `/v3/projects/4979012aca2645e2b094cf2062f6a711/users/917fad5bf9a14b19aa436f65f9bd36f1/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.760078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-50718596-ee61-461f-adc5-6aa7a4cdc33e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:28.761437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-50718596-ee61-461f-adc5-6aa7a4cdc33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.779089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 859/1732] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:28.780726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30626-881a-4b4c-9dba-04191b355850 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.783552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30626-881a-4b4c-9dba-04191b355850 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.783847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30626-881a-4b4c-9dba-04191b355850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.802768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4979012aca2645e2b094cf2062f6a711, user_id=917fad5bf9a14b19aa436f65f9bd36f1, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.810218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.831433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7ff30626-881a-4b4c-9dba-04191b355850 None None] MFA Rules not processed for user `2cd91f6976174fbf9692e0973435a4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:28.837049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7ff30626-881a-4b4c-9dba-04191b355850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.919148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 874/1733] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:28 2022] PUT /identity/v3/projects/4979012aca2645e2b094cf2062f6a711/users/917fad5bf9a14b19aa436f65f9bd36f1/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:28.919148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.922262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.998680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 860/1734] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:29.000170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765a43-d805-47de-a634-6986b7cc2b88 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.000457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765a43-d805-47de-a634-6986b7cc2b88 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.000731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765a43-d805-47de-a634-6986b7cc2b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.081047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3e765a43-d805-47de-a634-6986b7cc2b88 None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:29.086069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3e765a43-d805-47de-a634-6986b7cc2b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.245871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9654ac-ce6a-4a76-8f67-20084ee07797 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:29.269049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:29.269765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.270051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.270362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] PATH_INFO: `/v3/projects/4a1b0f7b617342e0bd291bdfdedcfc5b/users/b38e318e8fe644ceb4cfd61c16b1d6c8/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.313316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:29.321795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1b0f7b617342e0bd291bdfdedcfc5b, user_id=b38e318e8fe644ceb4cfd61c16b1d6c8, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:29.324589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88d1e-3851-4267-9f2e-517986b8dedf admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:29.325024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 861/1735] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Aug 24 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:29.326923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.327417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.328014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.357504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:29.359237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.370740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 862/1736] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:29.370740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:29.373711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.392150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df187ef1-27ad-4c7f-9dcb-89ed6a34365e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:29.400012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:29.402451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.402451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.402451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.405383 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.424467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:29.434771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017080f-9ef1-417f-90d2-da85ee34fee8 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:29.463599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 863/1737] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:29.465006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 875/1738] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:28 2022] PUT /identity/v3/projects/4a1b0f7b617342e0bd291bdfdedcfc5b/users/b38e318e8fe644ceb4cfd61c16b1d6c8/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:29.480035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.480035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.480035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.480455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.481179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.481471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.505340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] MFA Rules not processed for user `5171bf0514c3474aae358a9d7cfaf538`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:29.506722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.527625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] MFA Rules not processed for user `917fad5bf9a14b19aa436f65f9bd36f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:29.529016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:29.653394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 876/1739] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:29.654300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:29.730551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:29.789747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 864/1740] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:29.791669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:29.870609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec59fe-8d80-4534-869e-aa7a77b93dd9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:29.882386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:29.883757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.884036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.884307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.904731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:29.906510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39727f9-d0b1-43d9-940a-03cb1aec8bb4 service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:29.929835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 877/1741] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:29.932128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.932422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.933507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.952251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f572ef-4892-4594-b402-aa5100894c48 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:29.963725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:29.964353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:29.964630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:29.964899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:29.968373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:29.968373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:29.969942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.008373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 865/1742] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:30.009380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.009746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] MFA Rules not processed for user `b38e318e8fe644ceb4cfd61c16b1d6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:30.011418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:30.026849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b2f8-9185-40b4-acee-a9d4afdf3706 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.040310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.040951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.041234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.042978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.044617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.048954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.269072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 866/1743] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:30 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:30.270183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.335145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 878/1744] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:30.336227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.385734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a8a95-0bef-4eee-a61a-c783557dd47f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.511203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70123dc7-7f9f-4786-b2c1-692019ca0c4c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.512833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.513843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.514337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.514848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.516368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.518928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.532200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.532833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.533345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.533345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.534195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.547672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.672086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 879/1745] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:30 2022] POST /identity/v3/users => generated 299 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:30.675146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.711349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 867/1746] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:30 2022] POST /identity/v3/projects => generated 361 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:30.715401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.734901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27ffa-0a71-482a-8378-4b1fa962a446 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.866332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a54b-9ab8-4b55-bfe5-244013d3a362 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.874937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.877573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.877573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.877573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.877573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.884622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.895035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.896507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.896772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.897047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.897713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.899615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:30.910971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 880/1747] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:30.916458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:30.930743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7033e-df0e-4544-ac7e-69e3a8d9ee33 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:30.935932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:30.936843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:30.937098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:30.937375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] PATH_INFO: `/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc/users/a4da2b5b870d4212815c1b1a75d553d1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:30.958889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3efc390c5f48a281d76cd2d082f8fc, user_id=a4da2b5b870d4212815c1b1a75d553d1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:30.963452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.035599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 868/1748] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:30 2022] POST /identity/v3/users => generated 281 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:31.038644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.050203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef895-ac53-4232-8285-76127210e78f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.074437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 881/1749] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:30 2022] PUT /identity/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc/users/a4da2b5b870d4212815c1b1a75d553d1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:31.090856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.156824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:31.272666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.274332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.275000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.276437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.280634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.283939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.313179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04f02-a150-4c0b-b6ff-6c0d0e0e9d36 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.322803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 869/1750] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:31.326555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.328563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.334072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.335992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.336736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.337530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.342704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.355917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d82142-8124-47eb-b085-a963114605a6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.373602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.376495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.380109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.380109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.380109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.386573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.395022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 882/1751] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:31.402865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.444035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b44e3-9c54-4a94-91c2-2a5206189626 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.460772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 870/1752] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:31.464438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.465491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.465932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.466588 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] PATH_INFO: `/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc/users/a4da2b5b870d4212815c1b1a75d553d1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.467034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.470542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3efc390c5f48a281d76cd2d082f8fc, user_id=a4da2b5b870d4212815c1b1a75d553d1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.474172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf54fe1-fa00-4025-b673-e4e49ced3861 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.505827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a37eb-a1cb-47df-9775-eb13f6de3ec4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.544511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 883/1753] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:31 2022] PUT /identity/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc/users/a4da2b5b870d4212815c1b1a75d553d1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:31.550499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.550837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.551802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.602196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] MFA Rules not processed for user `a4da2b5b870d4212815c1b1a75d553d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:31.603882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:31.790999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:31.804186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.805146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.805530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.805916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] PATH_INFO: `/v3/projects/67af2d743dfb44229990a3c6a49ab3cc/users/a85ffd74e212403688e2b74aa9f288f4/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.815703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 884/1754] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:31.816617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.823142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:31.886469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=67af2d743dfb44229990a3c6a49ab3cc, user_id=a85ffd74e212403688e2b74aa9f288f4, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.898351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:32.012137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 871/1755] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:31 2022] PUT /identity/v3/projects/67af2d743dfb44229990a3c6a49ab3cc/users/a85ffd74e212403688e2b74aa9f288f4/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:32.013410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:32.016936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:32.090012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddc929-5e5e-4cba-b4bb-cba97cc27d98 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:32.171962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f7e50-847e-4c53-adf0-0a7f94ddf7ad admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:32.182204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:32.201647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:32.201647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:32.201647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:32.326096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:32.328332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:32.330208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:32.332148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:32.401362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:32.411030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6084-ad2b-4053-aae0-455a6b82f7e6 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:32.448676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 872/1756] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:32.450538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:32.451023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:32.451446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:32.507426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] MFA Rules not processed for user `a85ffd74e212403688e2b74aa9f288f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:32.522331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:32.526882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:32.530367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:32.553470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 885/1757] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:32.790193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 873/1758] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:32.796717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.001166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.092634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2bf7e-8299-4bcf-9a82-c1fd2f13d66f service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:33.147080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:33.148193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.157209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.157479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.158598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:33.174396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:33.242955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6517099-1876-40c4-a755-cd6f6acb01ec None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:33.267083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:33.267551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.273379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.275455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] PATH_INFO: `/v3/users/b38e318e8fe644ceb4cfd61c16b1d6c8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.292025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38e318e8fe644ceb4cfd61c16b1d6c8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:33.295558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:33.380909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 886/1759] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:32 2022] POST /identity/v3/projects => generated 365 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:33.386407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.654085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18304c-5257-40a4-994d-e711b02f3f9d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:33.681049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:33.685082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.688016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.688526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.689540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38e318e8fe644ceb4cfd61c16b1d6c8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.692481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:33.694830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:33.824119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38e318e8fe644ceb4cfd61c16b1d6c8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.837659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 887/1760] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:33 2022] POST /identity/v3/users => generated 289 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:33.839416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.861244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38e318e8fe644ceb4cfd61c16b1d6c8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.864530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38e318e8fe644ceb4cfd61c16b1d6c8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.885027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cbbd5-49e6-451c-8838-ede451e6292e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:33.910014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:33.918103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.919094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.919946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.920860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:33.933663 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:33.942631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cb8953a-c018-4d33-9bc4-d9bc291f846c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38e318e8fe644ceb4cfd61c16b1d6c8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.963106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 874/1761] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:32 2022] DELETE /identity/v3/users/b38e318e8fe644ceb4cfd61c16b1d6c8 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:33.963305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 888/1762] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:33.966722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.967417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.968080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.972183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.975687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.005130 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae0d5-cf47-4ba8-b551-cb6760c01445 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.011284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:34.014247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.099060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:34.143623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 875/1763] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:34.157609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.159603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.171398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.172272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.177118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.177118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] PATH_INFO: `/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1/users/4ef58e8c02b64b41abf18e4eaa2d0b82/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.198559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaec805c774c4f4b8bc5fc31dc94f4f1, user_id=4ef58e8c02b64b41abf18e4eaa2d0b82, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.202418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.319335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 889/1764] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:33 2022] PUT /identity/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1/users/4ef58e8c02b64b41abf18e4eaa2d0b82/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:34.334237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.403267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a1247-ddf1-4643-aaca-cdf58d15a93c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.541056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac05a6-eca8-4ee0-a9c6-72ec299d802c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.553424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.554094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.554378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.554681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.555366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.558260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:34.560704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.584842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 890/1765] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:34.589276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.609718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.610772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.611226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.612180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.622358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef073f-88d2-42db-919c-454d298372fe admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.624922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.626803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953f271-a6ba-4ecd-a9d3-6cb221bf6ada service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.638399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.643136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.643437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.643724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] PATH_INFO: `/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1/users/4ef58e8c02b64b41abf18e4eaa2d0b82/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.649018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaec805c774c4f4b8bc5fc31dc94f4f1, user_id=4ef58e8c02b64b41abf18e4eaa2d0b82, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.649601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 876/1766] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:34.657811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.686697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 891/1767] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:34 2022] PUT /identity/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1/users/4ef58e8c02b64b41abf18e4eaa2d0b82/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:34.698018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.698740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.699210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.772288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] MFA Rules not processed for user `4ef58e8c02b64b41abf18e4eaa2d0b82`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:34.774541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.971149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 877/1768] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:35.014477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:35.016725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:35.171692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420395b4-2bd4-47c0-803d-b78d119c0847 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:35.195790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:35.195790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:35.198264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:35.198600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.221034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:35.224180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:35.240380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 892/1769] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:35.320467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:35.322761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:35.335704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a4aa5-1d7f-4be1-8d7d-d22be7376822 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:35.341825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:35.342645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:35.342977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:35.343251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.354284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:35.599447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:35.602409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:35.623155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 878/1770] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:35.766344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:35.768145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.032690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814e153-81a0-45d9-9ff5-12e35add0e38 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:36.041200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:36.056368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a11b4-8f66-4cb8-907d-684882e5f2f0 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:36.061821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:36.064057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.064990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:36.065582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.066472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.067073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] PATH_INFO: `/v3/projects/4a1b0f7b617342e0bd291bdfdedcfc5b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.068269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.068784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.069703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1b0f7b617342e0bd291bdfdedcfc5b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:36.076493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:36.077643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.221167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:36.226216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaa2d3-2392-4440-8949-2a031dca5bcc service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:36.260246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 879/1771] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:36.374923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1b0f7b617342e0bd291bdfdedcfc5b', 'request_id': 'req-db28e00a-c0e0-489a-a31e-b8e29f6e80de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.398751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1b0f7b617342e0bd291bdfdedcfc5b', 'request_id': 'req-db28e00a-c0e0-489a-a31e-b8e29f6e80de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.523040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1b0f7b617342e0bd291bdfdedcfc5b', 'request_id': 'req-db28e00a-c0e0-489a-a31e-b8e29f6e80de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.538633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.539671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.540204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.545536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] The token cache is being invalidate because project 4a1b0f7b617342e0bd291bdfdedcfc5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:36.545828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.546893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-db28e00a-c0e0-489a-a31e-b8e29f6e80de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.551179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 893/1772] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:35 2022] DELETE /identity/v3/projects/4a1b0f7b617342e0bd291bdfdedcfc5b => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:36.556541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40d06-12ce-465a-ba61-83a0c3f0fcd5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.557701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40d06-12ce-465a-ba61-83a0c3f0fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.557977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40d06-12ce-465a-ba61-83a0c3f0fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.594251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-57d40d06-12ce-465a-ba61-83a0c3f0fcd5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:36.595723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-57d40d06-12ce-465a-ba61-83a0c3f0fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.608327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:36.609817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.732561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 894/1773] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:36.741072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.741498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.741785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.782400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:36.784573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.809432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 895/1774] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:36.813624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:36.822533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 880/1775] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:36.828468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ef37-eb7c-46c2-8c3a-d0566ad046a0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:36.837395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:36.838884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.839314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.839726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.840727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:36.841173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:36.843066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:36.857650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 896/1776] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:36.862306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:36.891561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3209-c798-428a-9098-665c8b1e5ab2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:36.937450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:36.937450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.937450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.937450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.938314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:36.938314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.128386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 881/1777] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:36 2022] POST /identity/v3/projects => generated 371 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:37.130171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.149112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d928fe-9879-4a4d-a650-e89012938071 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.339286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.340629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.343074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.343541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.344401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.351597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.477219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:37.506907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.507993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 897/1778] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:37 2022] POST /identity/v3/users => generated 286 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:37.526114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f258e22-e515-4a1c-8b36-b579b86f4029 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.528253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.549037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:37.564333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.579258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.579258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.579258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.579258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.599987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.634600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 882/1779] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:37.643336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.667484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf576-85b4-4fe3-9e00-2b68408f09a2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.690189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.690974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.691784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.694171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.694947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.697333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.724120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 883/1780] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:37.738380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a51099-4618-4033-b5e4-8c19c042194f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.741654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.756036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.757109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.759187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.762874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.778883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f509f0-a90c-4d6f-b6d3-5dd6893f5201 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.797088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.798097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.802614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.802614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] PATH_INFO: `/v3/projects/0767684cf0e64d1aa0a337d55a705433/users/167b45b560364793ba924fc2bbeeafd5/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.820673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0767684cf0e64d1aa0a337d55a705433, user_id=167b45b560364793ba924fc2bbeeafd5, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.825433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.864359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.866458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e8e3a-b82b-4ff1-ad10-facbe3625dd9 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.914677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 898/1781] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:56:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:56:37.936549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 884/1782] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:37 2022] PUT /identity/v3/projects/0767684cf0e64d1aa0a337d55a705433/users/167b45b560364793ba924fc2bbeeafd5/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:37.949561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.950006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.950411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.088271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] MFA Rules not processed for user `167b45b560364793ba924fc2bbeeafd5`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:38.093344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:38.143895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:38.481740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:38.525532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 899/1783] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 577 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:38.529474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:38.748143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bc6c3-e008-43d2-a0a1-b168ca096ed4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:38.788348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:38.789101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.789592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.790075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.791321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:38.808618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.071922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 885/1784] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:38 2022] POST /identity/v3/projects => generated 373 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:39.075882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.095773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7933dd-e718-40e7-87a1-bbaae70280f5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.318972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.326153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.326153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.326153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.327875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.330132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.468038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 900/1785] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:39 2022] POST /identity/v3/users => generated 293 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:39.473053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.525960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8390-bf9b-42d4-8ee1-474faa565d87 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.564975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.567288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.576441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.576935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.578271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.593541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.683574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 886/1786] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:39.685138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.747480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c668-ca14-49dd-b3d9-62daab18af4a admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.747480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.749084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.749376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.749893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] PATH_INFO: `/v3/projects/cdbb1aa9977147919cc5fbb1282fb808/users/a250cbd0e72a487f87166ed7bf6d272c/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.868068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbb1aa9977147919cc5fbb1282fb808, user_id=a250cbd0e72a487f87166ed7bf6d272c, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.876233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93ea1f-f239-4f1e-9c34-fd49c2bcb849 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.014383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 901/1787] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:39 2022] PUT /identity/v3/projects/cdbb1aa9977147919cc5fbb1282fb808/users/a250cbd0e72a487f87166ed7bf6d272c/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:40.021082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.055858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc3583-175e-4316-9676-dfb48546b10c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.193289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.194360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.194919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.256859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:40.263250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.410514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.411649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.412066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.412553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.413831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.421702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.469322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 902/1788] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 24 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:40.494348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 887/1789] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:40.500419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.522277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31790bb-651c-4987-8dce-182c2a567946 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.537716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.541334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.542040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.543514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] PATH_INFO: `/v3/projects/cdbb1aa9977147919cc5fbb1282fb808/users/a250cbd0e72a487f87166ed7bf6d272c/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.549434 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:40.553140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbb1aa9977147919cc5fbb1282fb808, user_id=a250cbd0e72a487f87166ed7bf6d272c, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.565581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb97e6e-cb68-49e5-9109-72179d179253 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.574758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.578394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.643144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f9b20-a961-48e9-beb0-c55dd0960e48 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.650910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 903/1790] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:40 2022] PUT /identity/v3/projects/cdbb1aa9977147919cc5fbb1282fb808/users/a250cbd0e72a487f87166ed7bf6d272c/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:40.658763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.660389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.661345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.713281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] MFA Rules not processed for user `a250cbd0e72a487f87166ed7bf6d272c`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:40.715706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.723895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:40.959578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 904/1791] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:40.984474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.985112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.985878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.986293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.002854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.006359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.024911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 888/1792] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:41.040132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.041694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.085152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7a04d-7d92-4f31-a684-8b99797edc50 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.103451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.103451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.103451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.103451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.141992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.149323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.155591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.159556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.178048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 905/1793] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:41.303834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828fbdc-c38b-4ef3-a333-14a8557d6884 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.310683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.311968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.312854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.314696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.332870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.332870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.354496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 889/1794] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:42.008857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.036828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdacfa8-1976-4f00-9a0d-5c883a6411da service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.059614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.060546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.060831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.061324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.276597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.279589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.317218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 906/1795] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:42.648694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:42.971898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:43.304561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.306004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.461816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e903-328f-4a03-bd9a-fd9a1241feaa service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.467842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.469682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.470156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.470593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] PATH_INFO: `/v3/users/a85ffd74e212403688e2b74aa9f288f4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.472183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85ffd74e212403688e2b74aa9f288f4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.474344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.709325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85ffd74e212403688e2b74aa9f288f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.819916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85ffd74e212403688e2b74aa9f288f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.840804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85ffd74e212403688e2b74aa9f288f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.848376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85ffd74e212403688e2b74aa9f288f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.902014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e64a9a91-d04d-4b2a-bc11-329485b9dbee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85ffd74e212403688e2b74aa9f288f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.924014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.928799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:43.954771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 890/1796] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:43 2022] DELETE /identity/v3/users/a85ffd74e212403688e2b74aa9f288f4 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:43.958193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1dff-f258-41bf-bb96-43e0fd4c4ecf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.961917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ec751-3ec4-4bd0-9e08-3f8decc4aacf service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.975042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1dff-f258-41bf-bb96-43e0fd4c4ecf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.980230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1dff-f258-41bf-bb96-43e0fd4c4ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.079269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-66dc1dff-f258-41bf-bb96-43e0fd4c4ecf None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:44.088795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-66dc1dff-f258-41bf-bb96-43e0fd4c4ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.193189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.193189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.193189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.193189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.194730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.334542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 891/1797] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:44.361407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.371182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.371182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.371182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 892/1798] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:56:44.381610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.386704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.415904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.422099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b702e-01f1-4a47-80ef-25ec8c19d086 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.472684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 907/1799] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:44.474305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8665-524f-4415-8893-e069e18c349c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.474599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8665-524f-4415-8893-e069e18c349c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.474919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8665-524f-4415-8893-e069e18c349c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.549453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-289d8665-524f-4415-8893-e069e18c349c None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:44.555683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-289d8665-524f-4415-8893-e069e18c349c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.584379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360de4df-a415-4d24-9b00-2d6c7a8df626 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.588889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.589618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.590039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.590413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.606360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.610415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.643441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 893/1800] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:44.747328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:44.814770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 908/1801] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:45.716046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.718692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:45.812085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284afd1-5497-49be-b9ee-3f16e30c3652 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.827555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.828535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.828983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.829405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.851165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.853093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d1859-8760-4f01-af8e-49224c08527b service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.878011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 894/1802] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:46.432933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0cfe-46a7-49a7-9d26-82ff0e90d4dd None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.433524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0cfe-46a7-49a7-9d26-82ff0e90d4dd None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.434065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0cfe-46a7-49a7-9d26-82ff0e90d4dd None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.442003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 909/1803] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:56:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:46.476452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcb83c-8a00-404d-931e-82392b5542c4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.477019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcb83c-8a00-404d-931e-82392b5542c4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.477444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcb83c-8a00-404d-931e-82392b5542c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.513129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a3dcb83c-8a00-404d-931e-82392b5542c4 None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:46.514732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a3dcb83c-8a00-404d-931e-82392b5542c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:46.544937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 895/1804] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:46.721214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8150-e40d-4f6a-a226-a39ae570aa94 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.722625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8150-e40d-4f6a-a226-a39ae570aa94 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.723496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8150-e40d-4f6a-a226-a39ae570aa94 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.727525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 910/1805] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:56:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:46.746005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.747342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.748917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.784561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:46.786236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:46.916389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb8150-e40d-4f6a-a226-a39ae570aa94 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.979774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb8150-e40d-4f6a-a226-a39ae570aa94 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.000965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 896/1806] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:47.005477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.006204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.006716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.007160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.048822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.052271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.086287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 911/1807] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:47.343835 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.345668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.371568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb944-2951-45df-b751-5d6937930098 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.386098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.388706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.389268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.390636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.438447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.443884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.473079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 897/1808] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:47.531212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:47.965765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.968697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:48.044142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17e8f1-7c68-487a-95f7-d27d9a28af74 service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.064342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.065880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.066368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.066864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] PATH_INFO: `/v3/projects/67af2d743dfb44229990a3c6a49ab3cc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.075347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67af2d743dfb44229990a3c6a49ab3cc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.076969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:48.242008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:48.357996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67af2d743dfb44229990a3c6a49ab3cc', 'request_id': 'req-fff7d631-5908-47d8-a38c-4bddbf407520'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.375311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67af2d743dfb44229990a3c6a49ab3cc', 'request_id': 'req-fff7d631-5908-47d8-a38c-4bddbf407520'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.488271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67af2d743dfb44229990a3c6a49ab3cc', 'request_id': 'req-fff7d631-5908-47d8-a38c-4bddbf407520'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.523766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] The token cache is being invalidate because project 67af2d743dfb44229990a3c6a49ab3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:48.524315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.528510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.532552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 912/1809] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:47 2022] DELETE /identity/v3/projects/67af2d743dfb44229990a3c6a49ab3cc => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:48.542116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:48.559028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af959-c542-4e69-ad3b-aec5f0c2c927 service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.735079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.736361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.739774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.740158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] PATH_INFO: `/v3/users/4ef58e8c02b64b41abf18e4eaa2d0b82` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.743654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef58e8c02b64b41abf18e4eaa2d0b82)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.750549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:49.089305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef58e8c02b64b41abf18e4eaa2d0b82'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.202873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef58e8c02b64b41abf18e4eaa2d0b82'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.215015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef58e8c02b64b41abf18e4eaa2d0b82'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.216176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef58e8c02b64b41abf18e4eaa2d0b82'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.255680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef58e8c02b64b41abf18e4eaa2d0b82'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.267217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 898/1810] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:48 2022] DELETE /identity/v3/users/4ef58e8c02b64b41abf18e4eaa2d0b82 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:49.907390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:49.909609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:49.923460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d631-5908-47d8-a38c-4bddbf407520 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:49.954379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:49.956822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:49.994346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:50.019195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.020397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.020397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.020894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] PATH_INFO: `/v3/users/e14b06fc52c049a197466bb83dd88f05` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.022249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14b06fc52c049a197466bb83dd88f05)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.024437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.068980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d40a-e746-4cfc-bf9d-c59ea74d4c47 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.074105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.074888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.075241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.075588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] PATH_INFO: `/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.082960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaec805c774c4f4b8bc5fc31dc94f4f1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.082960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.278436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14b06fc52c049a197466bb83dd88f05'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.281230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:50.283533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14b06fc52c049a197466bb83dd88f05'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.295506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaec805c774c4f4b8bc5fc31dc94f4f1', 'request_id': 'req-20955611-1e53-4ec2-a45a-4e9a382d148c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.295506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaec805c774c4f4b8bc5fc31dc94f4f1', 'request_id': 'req-20955611-1e53-4ec2-a45a-4e9a382d148c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.421703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14b06fc52c049a197466bb83dd88f05'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.440464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14b06fc52c049a197466bb83dd88f05'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.456953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5d3ebd72-66c0-4a57-a5c5-9f1381dae835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14b06fc52c049a197466bb83dd88f05'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.464631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 913/1811] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:49 2022] DELETE /identity/v3/users/e14b06fc52c049a197466bb83dd88f05 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:50.530391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaec805c774c4f4b8bc5fc31dc94f4f1', 'request_id': 'req-20955611-1e53-4ec2-a45a-4e9a382d148c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.560370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] The token cache is being invalidate because project eaec805c774c4f4b8bc5fc31dc94f4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:50.562146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.563542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-20955611-1e53-4ec2-a45a-4e9a382d148c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.566310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 899/1812] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:49 2022] DELETE /identity/v3/projects/eaec805c774c4f4b8bc5fc31dc94f4f1 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:50.573145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.573515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.574052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.621576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:50.623428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.739507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 914/1813] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.749734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.750485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.751234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.779220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:50.779761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.789589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 900/1814] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.795098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.807878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61a643-e959-4c43-9e22-e8be10f4f32c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.813975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.814675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.815514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.815876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.816529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:50.816866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.818571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.832201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 915/1815] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:50.836283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.846445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ab88c-9445-4913-aefb-b9655a661d84 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.852268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.852898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.853179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.853450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.854115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.856429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.871552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.878831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.890125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ee015-f843-42ae-86ce-7f8bd67c1815 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.897188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.898255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.898255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.898477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] PATH_INFO: `/v3/projects/c90f5f603c1f4edeac6252e231f247ae` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.899660 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90f5f603c1f4edeac6252e231f247ae)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.903408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.056229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 901/1816] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:51.057162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.059303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.071298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90f5f603c1f4edeac6252e231f247ae', 'request_id': 'req-9f77ecb2-e40a-432f-a53d-6c4297e59a25'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.078243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90f5f603c1f4edeac6252e231f247ae', 'request_id': 'req-9f77ecb2-e40a-432f-a53d-6c4297e59a25'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.157168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273497a8-d4df-4134-a380-f7a3c9cd5b0d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.162525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.163201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.163476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.163748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] PATH_INFO: `/v3/users/4ad2ba3c966f44848a78b8185e61a505` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.164932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad2ba3c966f44848a78b8185e61a505)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.166931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.202445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90f5f603c1f4edeac6252e231f247ae', 'request_id': 'req-9f77ecb2-e40a-432f-a53d-6c4297e59a25'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.224902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] The token cache is being invalidate because project c90f5f603c1f4edeac6252e231f247ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:51.225733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.227398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.230411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 916/1817] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:50 2022] DELETE /identity/v3/projects/c90f5f603c1f4edeac6252e231f247ae => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:51.231998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.234295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.321619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ecb2-e40a-432f-a53d-6c4297e59a25 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.328251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.328903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.329184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.329466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.330469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.332489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.493349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad2ba3c966f44848a78b8185e61a505'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.494521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 917/1818] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:51 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:51.496147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.505783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.521018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b6ad9-b026-40ae-8244-9caaea3fe5fe admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.537649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.538562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.538876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.539158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] PATH_INFO: `/v3/users/4d1b5ef6c30249b28d1df8e7ded7a5e1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.544488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1b5ef6c30249b28d1df8e7ded7a5e1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.545809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.590463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad2ba3c966f44848a78b8185e61a505'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.606244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad2ba3c966f44848a78b8185e61a505'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.606244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad2ba3c966f44848a78b8185e61a505'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.639630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad2ba3c966f44848a78b8185e61a505'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.650744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 902/1819] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:51 2022] DELETE /identity/v3/users/4ad2ba3c966f44848a78b8185e61a505 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:51.652304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.656581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.806353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2831c-53cc-44fb-bd8d-c4fbf2f2d3c4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.827963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.829006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.829598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.830148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.832056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.835030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cbab7-2b12-4186-a579-5baea74bad44 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.855658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 903/1820] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:51.857205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.857493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.857765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.859196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1b5ef6c30249b28d1df8e7ded7a5e1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.860009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1b5ef6c30249b28d1df8e7ded7a5e1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.885352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:51.886577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.895729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 904/1821] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:51.911671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.911671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.921777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1dfdc-0013-4565-9314-c9096303dd9b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.926844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.927464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.927740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.928011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.928662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.930410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.947718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 905/1822] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:51.948687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.951902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.953943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1b5ef6c30249b28d1df8e7ded7a5e1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.963528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1b5ef6c30249b28d1df8e7ded7a5e1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.981415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1b5ef6c30249b28d1df8e7ded7a5e1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.994627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 918/1823] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:51 2022] DELETE /identity/v3/users/4d1b5ef6c30249b28d1df8e7ded7a5e1 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:51.996576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.011146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:52.117562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1536e2-f51c-4910-ab60-fab48869fdc5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.121479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad57a-9657-4b20-83f2-bf22d08cdcc0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.123796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.135439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.135439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.135439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] PATH_INFO: `/v3/projects/d274921882aa4770ac4eca4faeb8ad28/users/d409a2de6e5a477f94a48d2830cb84d7/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.150810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d274921882aa4770ac4eca4faeb8ad28, user_id=d409a2de6e5a477f94a48d2830cb84d7, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.154100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.210019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.210809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.211158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.211511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.231012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.234258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818f8e0-be69-4d93-ae39-bc093cb83f3f service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.250075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 906/1824] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:52.272284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 919/1825] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:51 2022] PUT /identity/v3/projects/d274921882aa4770ac4eca4faeb8ad28/users/d409a2de6e5a477f94a48d2830cb84d7/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:52.282813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.283145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.283355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.321377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] MFA Rules not processed for user `d409a2de6e5a477f94a48d2830cb84d7`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:52.322851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:52.473630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 907/1826] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:52.478316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.600389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.601924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:52.620000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774a8b-0e23-4fcd-84ee-937002d55bcd admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.637610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.638256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.638586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.638920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.639592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.642458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.659096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed534049-05dc-498a-99ef-a7139d519861 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.670186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.671732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.672410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.674191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] PATH_INFO: `/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.677574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e8fbf8752b418cbe4204f4b95f5ec9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.685855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.782721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 920/1827] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/projects => generated 377 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:52.783891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.947729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:52.951863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e8fbf8752b418cbe4204f4b95f5ec9', 'request_id': 'req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.967697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e8fbf8752b418cbe4204f4b95f5ec9', 'request_id': 'req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:53.050237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8c635-544f-47ff-802d-59fb7fae6ea6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.064560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e8fbf8752b418cbe4204f4b95f5ec9', 'request_id': 'req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:53.086277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] The token cache is being invalidate because project 11e8fbf8752b418cbe4204f4b95f5ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:53.086595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:53.087877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0e0c6e2d-f2d1-439f-a8b2-cc884849e799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:53.095218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 908/1828] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:52 2022] DELETE /identity/v3/projects/11e8fbf8752b418cbe4204f4b95f5ec9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:53.097393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7663-4f1e-41ba-b03e-cc9eb6137b3a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.099335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7663-4f1e-41ba-b03e-cc9eb6137b3a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.099749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7663-4f1e-41ba-b03e-cc9eb6137b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.155335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-402f7663-4f1e-41ba-b03e-cc9eb6137b3a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:53.160135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-402f7663-4f1e-41ba-b03e-cc9eb6137b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.265764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:53.275289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.277137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.279053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.279424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.281570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.287984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.371274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 909/1829] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:53.374796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.375392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.375886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.438767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:53.441942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.473295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 921/1830] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/users => generated 295 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:53.476263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.517159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.536367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d39874-2c25-4aa7-970e-047f4ab769bc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.540343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 910/1831] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:53.544354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.545664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.545664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.545664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.546361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.547876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.548272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.577051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 922/1832] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:53.587775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf879-e458-42d6-99c0-fb6f134e9c66 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.600597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.615260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.617157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.625901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.625901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.625901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:53.625901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.625901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.641842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5803c-0de9-4cb9-9f76-6011225fe5c1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.669942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.675451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.680439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 911/1833] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:53.682743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.685043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.687126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] PATH_INFO: `/v3/projects/df97dc3995f44b8688f45b033cebf582/users/b3d0aeebbe5740aa9040695be8f9c024/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.736313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e392f-abe2-4a37-9579-a7f9efb0032b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.756385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.757164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.757341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df97dc3995f44b8688f45b033cebf582, user_id=b3d0aeebbe5740aa9040695be8f9c024, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.757544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.757822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.758691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.761173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.769700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.892354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 912/1834] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:53.893239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 923/1835] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:53 2022] PUT /identity/v3/projects/df97dc3995f44b8688f45b033cebf582/users/b3d0aeebbe5740aa9040695be8f9c024/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:53.900151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.901237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.010837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:54.114735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0a418-a504-4825-bebb-e9cd41f013ef admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.120658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.121561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.122606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.123160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.124048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.126140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.141997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 913/1836] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:54.146227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.169935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e421f-744f-481e-a8d9-3a6cfee75e05 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.182945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.182945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.182945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.182945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] PATH_INFO: `/v3/projects/df97dc3995f44b8688f45b033cebf582/users/b3d0aeebbe5740aa9040695be8f9c024/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.189160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df97dc3995f44b8688f45b033cebf582, user_id=b3d0aeebbe5740aa9040695be8f9c024, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.211391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14abb-5396-41f8-98cb-d21fe4b4717e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.224505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702b4aa-51f9-4cc5-a960-6a789ec15c07 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.237973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.240606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.243312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.244142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.245134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.257693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.274858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 914/1837] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:54 2022] PUT /identity/v3/projects/df97dc3995f44b8688f45b033cebf582/users/b3d0aeebbe5740aa9040695be8f9c024/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:54.293222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.293519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.293766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.370772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] MFA Rules not processed for user `b3d0aeebbe5740aa9040695be8f9c024`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:54.379926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.405114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 924/1838] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/users => generated 292 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:54.412090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.488808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272f854-e32c-4d95-a3e1-38cea4e5ea45 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.586053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 915/1839] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:54.600649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.601994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.693439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.696085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.699611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.700388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.701879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.713408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.717638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:54.765448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 925/1840] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:54.766910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.781039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31a4b-8012-44c5-b36b-b97ce01c0357 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.799413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.799413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.799413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.799413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.800103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.805748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.874076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 926/1841] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:54.889532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.891537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b9ed-e612-42c6-a5fe-04f3f9b46d72 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.899751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.905704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.907261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.915735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd5c2a-b778-463f-b08d-7e91c540b107 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.918448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.926242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.929064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.930332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.931247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.931886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] PATH_INFO: `/v3/projects/800033793a4445928274b952454cdeca/users/59bb917799c14213b19f3ad91097be81/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.997307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=800033793a4445928274b952454cdeca, user_id=59bb917799c14213b19f3ad91097be81, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.004845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf374-f7b2-4637-a414-ac844d1f77ca admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:55.184369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.186085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:55.203473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 916/1842] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:55.208664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:55.208664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.244123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8f7c-5d92-45a1-8958-a360376c4958 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:55.255507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:55.256185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.256548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.256702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] PATH_INFO: `/v3/projects/065be83bf9f44e6ea20d7bddb44ac3f3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.258707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065be83bf9f44e6ea20d7bddb44ac3f3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.260799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:55.293203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 927/1843] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:54 2022] PUT /identity/v3/projects/800033793a4445928274b952454cdeca/users/59bb917799c14213b19f3ad91097be81/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:55.310029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.310323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.310575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.368785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] MFA Rules not processed for user `59bb917799c14213b19f3ad91097be81`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:55.370912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.442154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:55.537233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 928/1844] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:55.538518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:55.621563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065be83bf9f44e6ea20d7bddb44ac3f3', 'request_id': 'req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:55.635093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065be83bf9f44e6ea20d7bddb44ac3f3', 'request_id': 'req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:55.717257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41d422-df9b-4ba9-9158-7619038f0287 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:55.761609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065be83bf9f44e6ea20d7bddb44ac3f3', 'request_id': 'req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:55.782756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] The token cache is being invalidate because project 065be83bf9f44e6ea20d7bddb44ac3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:55.783081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:55.784413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:55.786122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 917/1845] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:55 2022] DELETE /identity/v3/projects/065be83bf9f44e6ea20d7bddb44ac3f3 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:55.791590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:55.795663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.857702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:55.858343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.858616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.858948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.859587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.863531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:55.936093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c23e4a-aab9-4d8c-874b-735d7b01aed5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:55.942693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:55.943290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.943541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.943808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] PATH_INFO: `/v3/users/917fad5bf9a14b19aa436f65f9bd36f1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.945032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=917fad5bf9a14b19aa436f65f9bd36f1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.947040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.069464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 929/1846] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:55 2022] POST /identity/v3/projects => generated 383 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:56.075547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.077718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:56.157600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be4d2c-841f-4bae-8abc-75bc8925af5c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.163068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.163725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.164014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.164313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.164954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.167026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.337511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917fad5bf9a14b19aa436f65f9bd36f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.363363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 930/1847] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/users => generated 298 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:56.372513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.399284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c436e-2b51-48d1-a70d-1cdbaf350c4d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.409514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.410641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.411249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.411817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.412949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.416350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.445161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 931/1848] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:56.449740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.463431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1379f82-61cc-4113-b21c-efd72c1ad255 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.470216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.470989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.471279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.471596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] PATH_INFO: `/v3/projects/562d8353bd9d4d23a1b56e079bacba19/users/103949871a814cf0b195ad9c007c3f71/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.479843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917fad5bf9a14b19aa436f65f9bd36f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.492901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917fad5bf9a14b19aa436f65f9bd36f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.494356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917fad5bf9a14b19aa436f65f9bd36f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.516974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562d8353bd9d4d23a1b56e079bacba19, user_id=103949871a814cf0b195ad9c007c3f71, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.522489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.529040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917fad5bf9a14b19aa436f65f9bd36f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.533311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 918/1849] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:55 2022] DELETE /identity/v3/users/917fad5bf9a14b19aa436f65f9bd36f1 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:56.651168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 932/1850] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:56 2022] PUT /identity/v3/projects/562d8353bd9d4d23a1b56e079bacba19/users/103949871a814cf0b195ad9c007c3f71/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:56.652581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.654784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:56.755434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be880ad-3995-4cfc-997b-e2b70bf4dc84 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.842279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.843354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.843354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.844374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.844565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.846202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1c526-1505-4eca-bc46-cd792382da5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.863224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.863669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 919/1851] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:56.876060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a63aa0-d3a2-47be-9c0e-d11c015eb411 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.884031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.885417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.885885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.886344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] PATH_INFO: `/v3/projects/562d8353bd9d4d23a1b56e079bacba19/users/103949871a814cf0b195ad9c007c3f71/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.890133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=562d8353bd9d4d23a1b56e079bacba19, user_id=103949871a814cf0b195ad9c007c3f71, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.894337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.907908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 933/1852] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:56 2022] PUT /identity/v3/projects/562d8353bd9d4d23a1b56e079bacba19/users/103949871a814cf0b195ad9c007c3f71/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:56.920355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.920657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.920922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.934179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:56.963413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] MFA Rules not processed for user `103949871a814cf0b195ad9c007c3f71`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:56.964681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.109617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 920/1853] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:57.123923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.134253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.223821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb9e6e-7331-48bc-b88c-a55d153d7e5e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.228380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.229676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.231070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.231366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.234090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.306915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.308409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.320576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 934/1854] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:57.402065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.404203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.515403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a79932-f639-47fe-a381-5d1f693feaf6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.522083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.523293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.523646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.523999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] PATH_INFO: `/v3/projects/4979012aca2645e2b094cf2062f6a711` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.525434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4979012aca2645e2b094cf2062f6a711)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.530101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.579779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:57.680252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4979012aca2645e2b094cf2062f6a711', 'request_id': 'req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:57.686739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4979012aca2645e2b094cf2062f6a711', 'request_id': 'req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:57.764432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4979012aca2645e2b094cf2062f6a711', 'request_id': 'req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:57.782259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] The token cache is being invalidate because project 4979012aca2645e2b094cf2062f6a711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:57.782783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:57.784425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:57.787371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 921/1855] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:57 2022] DELETE /identity/v3/projects/4979012aca2645e2b094cf2062f6a711 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:57.793332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.806860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925460ba-7c10-4471-8c12-d56d75254f89 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.879113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.879768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.880050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.880324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] PATH_INFO: `/v3/users/a4da2b5b870d4212815c1b1a75d553d1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.881458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4da2b5b870d4212815c1b1a75d553d1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.883580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:58.162741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4da2b5b870d4212815c1b1a75d553d1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.163988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4da2b5b870d4212815c1b1a75d553d1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.304015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4da2b5b870d4212815c1b1a75d553d1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.310605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4da2b5b870d4212815c1b1a75d553d1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.327343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcf7f856-3151-4fc8-8919-35afbda768be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4da2b5b870d4212815c1b1a75d553d1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.338304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 935/1856] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:57 2022] DELETE /identity/v3/users/a4da2b5b870d4212815c1b1a75d553d1 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:59.086958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.089144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.109106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffc69b-8967-4a00-87b4-4efdb1d99ebe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.172990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.174159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.174436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.174735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] PATH_INFO: `/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.175862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3efc390c5f48a281d76cd2d082f8fc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.177781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:59.198052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:59.365825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3efc390c5f48a281d76cd2d082f8fc', 'request_id': 'req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.373398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3efc390c5f48a281d76cd2d082f8fc', 'request_id': 'req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.488128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3efc390c5f48a281d76cd2d082f8fc', 'request_id': 'req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.511421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] The token cache is being invalidate because project 9e3efc390c5f48a281d76cd2d082f8fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:59.512138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.514238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b9fa949f-a3d4-40d4-aa23-c337337c8fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.516586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 922/1857] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:59 2022] DELETE /identity/v3/projects/9e3efc390c5f48a281d76cd2d082f8fc => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:59.525661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.526144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.526349 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.553187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:59.555944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.616914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:59.653375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 936/1858] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:59.664182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.664596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.665715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.702752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:59.705478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.732333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 923/1859] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:59.737985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.748504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5d61-8e37-480c-8305-abf5addd7626 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.754253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.755124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.755527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.755902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.756676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:59.757076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.758935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:59.783934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 937/1860] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:59.785884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.802562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c287df8-b04b-4557-8d3e-973f420cd5f2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.809213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.810137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.810487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.810894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.811968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.814507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:59.963664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 924/1861] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:59.968489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.980155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ceba-5c1a-45f2-b3d0-0723be4b6cc0 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.040434 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.041065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.041348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.041623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.042518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.044523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.153570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 938/1862] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:00.155012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.165899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6045b-508c-4337-9d59-3f76356915c7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.170912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.171546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.171825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.172126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.172792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.174502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.185663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 925/1863] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:00.190320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.201808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8a44c-7906-45fe-be13-806ddfce7a46 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.208199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.209078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.209528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.209968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.211379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.213980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22634-bb55-415f-a217-dcb1f8f5450d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.237666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 939/1864] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:00.240069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.256969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c738f-c6bc-4d2d-8d27-1b62174249ce admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.263674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.264588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.264982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.265431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] PATH_INFO: `/v3/projects/7268f76678cf43e7b4932c25498631e3/users/7cece2baa6cc4490ad7d889fdcfd5430/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.293466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7268f76678cf43e7b4932c25498631e3, user_id=7cece2baa6cc4490ad7d889fdcfd5430, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.296496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.487885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 926/1865] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:00 2022] PUT /identity/v3/projects/7268f76678cf43e7b4932c25498631e3/users/7cece2baa6cc4490ad7d889fdcfd5430/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:00.658894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.658894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.658894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.658894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] MFA Rules not processed for user `7cece2baa6cc4490ad7d889fdcfd5430`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:00.658894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:00.682962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 940/1866] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:00.689017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.777401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94d7-47e2-4d85-971f-afd25b60979f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.785939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.786942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.787296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.787640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.788407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.790505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:01.849045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 927/1867] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:00 2022] POST /identity/v3/projects => generated 379 bytes in 1160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:01.853719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:01.866578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f49d81-243f-4d8b-bdbe-29c744bffa72 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:01.944542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:01.945603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:01.946097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:01.946590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:01.947975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:01.951837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:02.339735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 941/1868] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:01 2022] POST /identity/v3/users => generated 290 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:02.346903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.359958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e782d1-3fbb-4159-b27f-d3c489b52df3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.366971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.367601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.367886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.368228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.368891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.370612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:02.382969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 928/1869] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:02.394353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.400986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.402855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:02.413632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633052-47cb-4b61-83bb-d680f9c3ad5e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.421959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.422789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.423196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.424220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.425384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.428539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:02.443386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 942/1870] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:02.448983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.467665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eca6c-97cc-46ef-a91f-ee6a3b957763 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.476899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.478103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.478883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.479458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] PATH_INFO: `/v3/projects/757ba938efd94de58a8934324516df04/users/e907056e9e0f4413a53f8b2bed4827f1/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.495705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877248f-3337-4bf7-b343-00b2d2cd483c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.502747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.503687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.504201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.504702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.505833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=757ba938efd94de58a8934324516df04, user_id=e907056e9e0f4413a53f8b2bed4827f1, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.508161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:02.511442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e5c34-86f7-407d-b4b6-ac69de5627cd admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:02.617763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.624684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:02.631263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 943/1871] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:02 2022] PUT /identity/v3/projects/757ba938efd94de58a8934324516df04/users/e907056e9e0f4413a53f8b2bed4827f1/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:02.642933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.643530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.644021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.659467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 929/1872] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:02.694312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] MFA Rules not processed for user `e907056e9e0f4413a53f8b2bed4827f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:02.696670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:02.815902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:02.843353 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 944/1873] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:02.848441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.862059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9aaca-5e9f-4dd1-afbd-58a79cc4ad89 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.964458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.965349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.965852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.966320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.967525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.971371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.094710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 930/1874] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:02 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:03.098856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:03.126940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84874732-51c6-4f57-913c-8c31e1ad8ac7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:03.281004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:03.281696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.282011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.284485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.285184 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:03.293395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.430930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:03.459016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 945/1875] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:03 2022] POST /identity/v3/users => generated 296 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:03.463332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:03.478390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78f51-7645-4d83-8e33-01e04562ce64 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:03.483431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:03.484292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.484609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.484888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.485757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:03.487618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.500143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 931/1876] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:03.504398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:03.516689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6320dd-83cc-46a9-a323-c3884dd618f9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:03.527840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:03.528747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.529035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.529314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] PATH_INFO: `/v3/projects/8e920d733e5a4209804b3dd0c214f0fa/users/dac4bb3b52fe40ca960783564ec6cfe0/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.547652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e920d733e5a4209804b3dd0c214f0fa, user_id=dac4bb3b52fe40ca960783564ec6cfe0, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:03.550411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.664790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 946/1877] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:03 2022] PUT /identity/v3/projects/8e920d733e5a4209804b3dd0c214f0fa/users/dac4bb3b52fe40ca960783564ec6cfe0/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:03.669111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:03.681405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aebed-b1e7-4f7a-b47f-43cd4eb4c63a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:03.767323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:03.768101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.768399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.768759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.769442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:03.771184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724409a-5f55-4f63-b4e8-eeb600ee4333 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.794646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:03.794906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 932/1878] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:03.807816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c33cd-af12-4da1-b4b4-ba27416c44a1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:03.812768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:03.813450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.814317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.814882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] PATH_INFO: `/v3/projects/8e920d733e5a4209804b3dd0c214f0fa/users/dac4bb3b52fe40ca960783564ec6cfe0/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.819227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e920d733e5a4209804b3dd0c214f0fa, user_id=dac4bb3b52fe40ca960783564ec6cfe0, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:03.821896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.841075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 947/1879] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:03 2022] PUT /identity/v3/projects/8e920d733e5a4209804b3dd0c214f0fa/users/dac4bb3b52fe40ca960783564ec6cfe0/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:03.860982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:03.862848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:03.862848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:03.933120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] MFA Rules not processed for user `dac4bb3b52fe40ca960783564ec6cfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:03.942038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:04.244966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 933/1880] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:04.259616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:04.262571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:04.317365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:04.329430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2c6e8-c2bd-4e93-9f0e-7ea1c8cb8710 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:04.333350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:04.333997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:04.334259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:04.334823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:04.346506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:04.349361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9ff8-cc2f-42f1-b446-f6af685da392 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:04.373843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 948/1881] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:04.590269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:04.603108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e6934-3de0-48d2-bc8d-4511aa981721 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:04.607639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:04.608372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:04.608654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:04.608929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:04.694839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:04.696279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75afc49-a87a-4837-ba17-2b726b5ed743 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:04.706820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 934/1882] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:05.337427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:06.095866 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:06.529850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:06.898155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:07.575614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:08.044980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:08.045151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:08.045527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:08.050056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 949/1883] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:57:08.058700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:08.059510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:08.059510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:08.080547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:08.081846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:08.170442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 935/1884] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:08.428497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:08.431024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:08.456935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0bc50-bf00-431d-8278-775c355f9f05 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:08.466558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:08.467410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:08.467795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:08.468243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:08.481833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:08.489926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:08.527664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 950/1885] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:08.604335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:08.735267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:09.555852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:09.573672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f5889-76c5-42cd-885b-8dd10f898fbd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:09.578327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:09.579076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:09.579406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:09.579691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:09.581556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:09.611305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:09.672269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:09.674412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef4545-6513-4d2a-808b-daa1b47be816 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:09.686062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 936/1886] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:09.750305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:11.092305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:11.226798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710539-bd28-42e7-886e-728ccfb3dead service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:11.234538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:11.235765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:11.236191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:11.236530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] PATH_INFO: `/v3/users/7cece2baa6cc4490ad7d889fdcfd5430` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:11.237628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cece2baa6cc4490ad7d889fdcfd5430)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:11.239613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:11.346569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:11.537313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cece2baa6cc4490ad7d889fdcfd5430'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:11.538185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cece2baa6cc4490ad7d889fdcfd5430'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:11.567789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:11.629300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cece2baa6cc4490ad7d889fdcfd5430'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:11.639965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cece2baa6cc4490ad7d889fdcfd5430'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:11.653482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cece2baa6cc4490ad7d889fdcfd5430'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:11.661696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 951/1887] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:11 2022] DELETE /identity/v3/users/7cece2baa6cc4490ad7d889fdcfd5430 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:11.666110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:11.666397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:11.666960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:11.687602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:11.688863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:11.751534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 937/1888] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:11.761374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:11.762851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:11.830192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8f4f3-9c91-43a5-96b3-9cdc5e1de20c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:11.835036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:11.835680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:11.835992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:11.836268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:11.851024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:11.851803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:11.864134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 952/1889] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:12.490717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:12.492264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:12.506209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1f6c4-dfa3-4fe4-be89-ef74502f940a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:12.511830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:12.512634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:12.513196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:12.513513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] PATH_INFO: `/v3/projects/7268f76678cf43e7b4932c25498631e3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:12.515103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7268f76678cf43e7b4932c25498631e3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:12.517159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:12.697921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7268f76678cf43e7b4932c25498631e3', 'request_id': 'req-8d038a0c-20ec-402b-ad94-4944756a89d3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:12.704683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7268f76678cf43e7b4932c25498631e3', 'request_id': 'req-8d038a0c-20ec-402b-ad94-4944756a89d3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:12.791967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7268f76678cf43e7b4932c25498631e3', 'request_id': 'req-8d038a0c-20ec-402b-ad94-4944756a89d3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:12.804920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] The token cache is being invalidate because project 7268f76678cf43e7b4932c25498631e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:12.805314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:12.805985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:12.807728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 938/1890] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:12 2022] DELETE /identity/v3/projects/7268f76678cf43e7b4932c25498631e3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:12.814031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:12.824591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f35c8-ca98-456f-882d-41345308121d service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:12.888367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:12.889020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:12.889328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:12.889612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] PATH_INFO: `/v3/users/e907056e9e0f4413a53f8b2bed4827f1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:12.890916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e907056e9e0f4413a53f8b2bed4827f1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:12.893627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:12.935350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:13.056527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e907056e9e0f4413a53f8b2bed4827f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.057172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e907056e9e0f4413a53f8b2bed4827f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.066345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:13.145864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e907056e9e0f4413a53f8b2bed4827f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.155548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e907056e9e0f4413a53f8b2bed4827f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.175937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e907056e9e0f4413a53f8b2bed4827f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.182563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 953/1891] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:12 2022] DELETE /identity/v3/users/e907056e9e0f4413a53f8b2bed4827f1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:13.832589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:13.834529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:13.849821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:13.853330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038a0c-20ec-402b-ad94-4944756a89d3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:13.937050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:13.937726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:13.937987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:13.938303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] PATH_INFO: `/v3/projects/757ba938efd94de58a8934324516df04` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:13.939603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=757ba938efd94de58a8934324516df04)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:13.941670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:13.959409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:14.072763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757ba938efd94de58a8934324516df04', 'request_id': 'req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.082700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757ba938efd94de58a8934324516df04', 'request_id': 'req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.160427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757ba938efd94de58a8934324516df04', 'request_id': 'req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.175915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] The token cache is being invalidate because project 757ba938efd94de58a8934324516df04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:14.176294 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.176973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.178938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 939/1892] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:13 2022] DELETE /identity/v3/projects/757ba938efd94de58a8934324516df04 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:14.183855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:14.187160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.187331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:14.209372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ff235-2fb3-41b8-84ae-33ca2b8d588e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:14.297862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:14.299404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.300518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.300959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] PATH_INFO: `/v3/users/dac4bb3b52fe40ca960783564ec6cfe0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.305626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac4bb3b52fe40ca960783564ec6cfe0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:14.310499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:14.675265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac4bb3b52fe40ca960783564ec6cfe0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.676411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac4bb3b52fe40ca960783564ec6cfe0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.838642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac4bb3b52fe40ca960783564ec6cfe0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.849894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac4bb3b52fe40ca960783564ec6cfe0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.868338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:14.869845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f60889c3-aceb-4d91-bf8b-e0c620da375c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac4bb3b52fe40ca960783564ec6cfe0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.877778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 954/1893] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:14 2022] DELETE /identity/v3/users/dac4bb3b52fe40ca960783564ec6cfe0 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:15.684548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:15.685046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:15.701179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d39ac-4bb1-41c5-99bb-e2cd81722dca admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:15.837673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:15.838392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:15.838808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:15.839137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] PATH_INFO: `/v3/projects/8e920d733e5a4209804b3dd0c214f0fa` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:15.840624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e920d733e5a4209804b3dd0c214f0fa)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:15.843420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.078370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e920d733e5a4209804b3dd0c214f0fa', 'request_id': 'req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.103382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e920d733e5a4209804b3dd0c214f0fa', 'request_id': 'req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.234917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e920d733e5a4209804b3dd0c214f0fa', 'request_id': 'req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.248871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] The token cache is being invalidate because project 8e920d733e5a4209804b3dd0c214f0fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:16.249170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.249842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e96d55c1-13fb-4e7e-bc5e-f15cee19f410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.251597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 940/1894] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:15 2022] DELETE /identity/v3/projects/8e920d733e5a4209804b3dd0c214f0fa => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:16.258841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.259402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.259546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.283533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.284921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.374718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 955/1895] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:16.382311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.383715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.383715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.415064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.417296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.425775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 941/1896] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:16.431383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.442553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344bb19-1d6f-4199-a639-6021f3ac4d1a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.464895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.465773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.466158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.467256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.467498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:16.467939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.471448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.497691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 956/1897] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:16.502330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.511900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cef46-fd5a-445c-b50f-abe3b0a1d462 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.516674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.517296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.517571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.517842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.524805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.533739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.646261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 942/1898] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:16.651294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.669943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f5ef4-0646-4b94-923b-72d0caaa17ad admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.698968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:16.732880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.733404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.733711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.733985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.735174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.737109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.875887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 957/1899] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:16.881131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.907196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f629-8293-4d06-9963-a8f0431beb44 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.914574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.915792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.916330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.916909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.917763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.920008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.937411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 943/1900] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:16.939654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.959632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1a6b-4964-4163-b5c0-b6c5231b75f4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.968058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.970593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.970593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.970593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.971399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.973343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0b511-e973-44d4-8979-0f896b37ff56 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.989609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 958/1901] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:16.994449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.006715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202153a-d0fd-481f-b4b0-b258c2b86203 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.017380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.018091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.018370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.018745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] PATH_INFO: `/v3/projects/107e0ce164854464ab37876e87062708/users/f2183b79d5e946f0a444ded3b7747996/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.046485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107e0ce164854464ab37876e87062708, user_id=f2183b79d5e946f0a444ded3b7747996, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.051272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.199409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 944/1902] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:16 2022] PUT /identity/v3/projects/107e0ce164854464ab37876e87062708/users/f2183b79d5e946f0a444ded3b7747996/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:17.212941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.213962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.214874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.318200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] MFA Rules not processed for user `f2183b79d5e946f0a444ded3b7747996`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:17.318200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.438837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 959/1903] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:17.495546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.495546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.527556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:17.579886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b59dbc-07bf-4318-adff-61ee462276e3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.583820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.584469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.584756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.585029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.595564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.597269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310b4ba-41ee-480d-8fb9-528107f484fe service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.607759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 945/1904] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:17.629232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.744062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb13e-2051-45cd-b5e4-9baf17ad6da9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.749754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.751243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.751684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.752053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] PATH_INFO: `/v3/users/f2183b79d5e946f0a444ded3b7747996` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.753455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2183b79d5e946f0a444ded3b7747996)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.756878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:18.095900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:18.108513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2183b79d5e946f0a444ded3b7747996'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:18.109291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2183b79d5e946f0a444ded3b7747996'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:18.217395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2183b79d5e946f0a444ded3b7747996'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:18.228298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2183b79d5e946f0a444ded3b7747996'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:18.243944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2183b79d5e946f0a444ded3b7747996'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:18.253070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 960/1905] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:17 2022] DELETE /identity/v3/users/f2183b79d5e946f0a444ded3b7747996 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:18.259906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:18.260758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:18.260758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:18.282923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:18.284200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:18.362405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 946/1906] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:18.370128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:18.371957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:18.452738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafb85f-7f87-4cf3-a17d-b65ff1cb981b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:18.458962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:18.460266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:18.460465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:18.460807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:18.473335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:18.475108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b11f-6d9b-4b27-b8c3-f8bb082a5752 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:18.489443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 961/1907] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:19.111025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:19.112978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:19.139168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e2b5d-756c-437d-a044-2b80b315e567 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:19.146207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:19.148334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.148334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.148518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] PATH_INFO: `/v3/projects/107e0ce164854464ab37876e87062708` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.152167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=107e0ce164854464ab37876e87062708)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:19.162156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:19.346482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107e0ce164854464ab37876e87062708', 'request_id': 'req-37c0d3df-dce3-4896-a287-781425fa7449'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:19.358014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107e0ce164854464ab37876e87062708', 'request_id': 'req-37c0d3df-dce3-4896-a287-781425fa7449'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:19.495975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107e0ce164854464ab37876e87062708', 'request_id': 'req-37c0d3df-dce3-4896-a287-781425fa7449'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:19.518834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] The token cache is being invalidate because project 107e0ce164854464ab37876e87062708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:19.520046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:19.521633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-37c0d3df-dce3-4896-a287-781425fa7449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:19.525475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 947/1908] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:19 2022] DELETE /identity/v3/projects/107e0ce164854464ab37876e87062708 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:19.535095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.536845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.539356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.574167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:19.575615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:19.646955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 962/1909] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:19.657840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.658598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.659102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.683329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:19.684788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:19.695208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 948/1910] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:19.699806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:19.710424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84be494-2876-4155-a4c2-21c9a9348406 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:19.715252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:19.715891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.716177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.716452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.717188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:19.717492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:19.719250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:19.732030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 963/1911] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:19.737189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:19.747239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d604a-376e-44f4-a3ae-69069def95b2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:19.753047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:19.754019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.754437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.754892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.755754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:19.757937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:19.856911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 949/1912] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:19 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:19.858328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:19.870528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8f54-a5a7-43a1-b2a0-e4de0e59e0d3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:19.916291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:19.938048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:19.938713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:19.939095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:19.939378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:19.940032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:19.942004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:20.051471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 964/1913] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:19 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:20.064339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:20.084790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbedeb9-06a4-405b-9349-bead72d2949b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:20.108066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:20.108738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.109025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.109858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.114575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:20.119996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:20.141388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 950/1914] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:20.145536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:20.157053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcae4b7-aa0c-4584-9e38-42d36ac57011 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:20.180248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:20.180248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.180248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.180248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.180600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:20.180600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a298ae-8a39-4378-a802-282bb36d07f0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:20.192299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 965/1915] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:20.196629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:20.204209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:20.211017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537fd0e-c1b6-47ad-a78b-0973a4db3d85 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:20.216176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:20.216870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.217146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.219912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] PATH_INFO: `/v3/projects/f8d0d6fedf3c41cebf8ea0df0dee4110/users/a7bdf6e175284604a1a1e4df6f628049/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.235770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d0d6fedf3c41cebf8ea0df0dee4110, user_id=a7bdf6e175284604a1a1e4df6f628049, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:20.238468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:20.344805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 951/1916] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:20 2022] PUT /identity/v3/projects/f8d0d6fedf3c41cebf8ea0df0dee4110/users/a7bdf6e175284604a1a1e4df6f628049/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:20.360646 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.363522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.363522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.405333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] MFA Rules not processed for user `a7bdf6e175284604a1a1e4df6f628049`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:20.407894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:20.588117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 966/1917] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:20.609366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:20.612380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:20.715076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3092-c602-4fe2-a39c-331f4763455b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:20.715076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:20.715076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.715076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.716007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.733949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:20.735822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4419d-2f24-4f37-a590-1dd771dde303 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:20.746921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 952/1918] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:20.776952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:20.873086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eee34-272c-4083-82a8-fedcbf453484 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:20.879627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:20.880954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:20.880954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:20.880954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] PATH_INFO: `/v3/users/a7bdf6e175284604a1a1e4df6f628049` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:20.882234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7bdf6e175284604a1a1e4df6f628049)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:20.884365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:21.127598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7bdf6e175284604a1a1e4df6f628049'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:21.128317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7bdf6e175284604a1a1e4df6f628049'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:21.277278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7bdf6e175284604a1a1e4df6f628049'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:21.279536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:21.288792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7bdf6e175284604a1a1e4df6f628049'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:21.313583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7bdf6e175284604a1a1e4df6f628049'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:21.324368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 967/1919] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:20 2022] DELETE /identity/v3/users/a7bdf6e175284604a1a1e4df6f628049 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:21.329143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:21.329431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:21.329723 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:21.350427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:21.351833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:21.441683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 953/1920] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:21.452544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:21.454223 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:21.524315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8587-7f81-4c9b-a9b8-2ba8220daa26 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:21.529620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:21.530415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:21.530758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:21.531614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:21.541788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:21.543276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3c9b-db75-469e-83db-06d618838219 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:21.556179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 968/1921] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:22.372757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:22.375778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:22.392279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa75287-8fc6-40af-9dd5-4116bc6d1536 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:22.408024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:22.408811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:22.409164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:22.409530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] PATH_INFO: `/v3/projects/f8d0d6fedf3c41cebf8ea0df0dee4110` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:22.411276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d0d6fedf3c41cebf8ea0df0dee4110)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:22.413415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:22.698301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d0d6fedf3c41cebf8ea0df0dee4110', 'request_id': 'req-cdca9f56-0045-41eb-8b70-90a24a22fc4c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:22.717621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d0d6fedf3c41cebf8ea0df0dee4110', 'request_id': 'req-cdca9f56-0045-41eb-8b70-90a24a22fc4c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:22.818690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d0d6fedf3c41cebf8ea0df0dee4110', 'request_id': 'req-cdca9f56-0045-41eb-8b70-90a24a22fc4c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:22.838206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] The token cache is being invalidate because project f8d0d6fedf3c41cebf8ea0df0dee4110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:22.839206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:22.840209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cdca9f56-0045-41eb-8b70-90a24a22fc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:22.851847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 954/1922] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:22 2022] DELETE /identity/v3/projects/f8d0d6fedf3c41cebf8ea0df0dee4110 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:22.860414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:22.862481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:22.862964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:22.896505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:22.897759 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.003394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 969/1923] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:23.016241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.016629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.016932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.028941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:23.046808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:23.048093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.059173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 955/1924] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:23.065678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.094402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020af1bf-310b-40bc-8739-b6431edb7b00 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.113839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.114944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.116436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.116700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.119025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:23.119025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.124342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.159724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 970/1925] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:23.161213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.177471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515533-fe35-46c9-9bf7-9e1cd80c1721 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.192826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.193467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.193746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.194018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.194690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.197011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.344094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 956/1926] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:23 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:23.348512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.362633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6156f6-9eaa-4894-9dd8-29586e0a4869 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.461310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.461956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.462240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.462521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.463373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.465904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.478062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:23.599141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 971/1927] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:23 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:23.603911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.620517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a980-a093-4754-b332-0c2e580f335c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.629561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.630297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.630586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.631012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.631607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.633364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.647531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 957/1928] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:23.652664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.674297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581385-cfc2-4ecc-ba23-cd6642753d49 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.681166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.681986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.682421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.682734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.683397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.685248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928e6b7-eab0-49e2-ba7a-75534cce992d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.699969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 972/1929] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:23.705793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.731199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc971-1e6a-401f-a248-c8f79e2afd27 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.740026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.741709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.742493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.743233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] PATH_INFO: `/v3/projects/12594f976e5f48d9926716737f86afe0/users/f1381dae2b774b48a590e21b3d30dbd3/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.773367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12594f976e5f48d9926716737f86afe0, user_id=f1381dae2b774b48a590e21b3d30dbd3, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.779360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.864401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 958/1930] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:23 2022] PUT /identity/v3/projects/12594f976e5f48d9926716737f86afe0/users/f1381dae2b774b48a590e21b3d30dbd3/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:23.869519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.870707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.870707 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.893742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:23.895317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.111471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.114359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.119018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 973/1931] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:24.210798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.213598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.249031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab011-4004-4807-b733-6bb4ff524dba None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.267320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.268073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.268364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.268643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] PATH_INFO: `/v3/users/59bb917799c14213b19f3ad91097be81` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.270357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bb917799c14213b19f3ad91097be81)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.273015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.296003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5f79-ad18-40b0-8be5-3de1c3686c0d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.300567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.301349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.301628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.301913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.314245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.316199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973bde9-61c2-4d54-969c-8b44f1171223 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.327162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 959/1932] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:24.568459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bb917799c14213b19f3ad91097be81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.569541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bb917799c14213b19f3ad91097be81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.677578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bb917799c14213b19f3ad91097be81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.693615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bb917799c14213b19f3ad91097be81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.722010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bb917799c14213b19f3ad91097be81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.738297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 974/1933] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:24 2022] DELETE /identity/v3/users/59bb917799c14213b19f3ad91097be81 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:24.739946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.740637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.740637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.764040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:24.765469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.864213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 960/1934] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:24.874724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.876924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.941508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b8598-32f9-4973-87ea-c33510aa8089 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.945382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.946014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.946301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.946579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.955382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:24.957831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.959383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.971158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 975/1935] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:25.797079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.800980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.812744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f97cd-ca92-48fd-8166-12b09309e83b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.825429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.826046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.826366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.826647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] PATH_INFO: `/v3/projects/800033793a4445928274b952454cdeca` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.829709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800033793a4445928274b952454cdeca)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.840659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.970108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:25.977240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800033793a4445928274b952454cdeca', 'request_id': 'req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.984481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800033793a4445928274b952454cdeca', 'request_id': 'req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.082838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800033793a4445928274b952454cdeca', 'request_id': 'req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.105130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] The token cache is being invalidate because project 800033793a4445928274b952454cdeca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:26.105441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.106195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.108513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 961/1936] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:25 2022] DELETE /identity/v3/projects/800033793a4445928274b952454cdeca => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:26.113360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.123532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58829-a25d-460c-ba1f-c142cec30c07 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.216732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.218368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.218368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.218966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] PATH_INFO: `/v3/users/103949871a814cf0b195ad9c007c3f71` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.220177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=103949871a814cf0b195ad9c007c3f71)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.222142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.549429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103949871a814cf0b195ad9c007c3f71'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.550480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103949871a814cf0b195ad9c007c3f71'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.658225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103949871a814cf0b195ad9c007c3f71'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.667996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103949871a814cf0b195ad9c007c3f71'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.682072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0be48ab2-0781-4731-a7b2-1590f931eee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103949871a814cf0b195ad9c007c3f71'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:26.689321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 976/1937] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:26 2022] DELETE /identity/v3/users/103949871a814cf0b195ad9c007c3f71 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:27.584980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:27.586542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:27.601743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190fc5-0312-45d2-9f0e-a7116fe6d1f3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.714643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.715852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.716441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.716902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] PATH_INFO: `/v3/projects/562d8353bd9d4d23a1b56e079bacba19` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.719852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=562d8353bd9d4d23a1b56e079bacba19)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.728685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.921195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562d8353bd9d4d23a1b56e079bacba19', 'request_id': 'req-2103e087-e151-4999-bf2f-c30f1a69ccc8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:27.929056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562d8353bd9d4d23a1b56e079bacba19', 'request_id': 'req-2103e087-e151-4999-bf2f-c30f1a69ccc8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:28.022912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562d8353bd9d4d23a1b56e079bacba19', 'request_id': 'req-2103e087-e151-4999-bf2f-c30f1a69ccc8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:28.037399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] The token cache is being invalidate because project 562d8353bd9d4d23a1b56e079bacba19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:28.037785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:28.038974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2103e087-e151-4999-bf2f-c30f1a69ccc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:28.040709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 962/1938] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:27 2022] DELETE /identity/v3/projects/562d8353bd9d4d23a1b56e079bacba19 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:28.048483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.049334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.049731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.079066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:28.080354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:28.179503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 977/1939] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:28.188054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.188395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.188645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.215000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:28.215243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:28.225856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 963/1940] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:28.231763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.252765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac03f1e-7c03-4185-8cd2-76c914943508 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.260816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.261891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.262377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.262917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.264212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:28.264709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.268084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.297240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 978/1941] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:28.301703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.313528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2e48b-0cec-4a5e-b114-95a4dd9cbdb5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.319600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.320391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.320739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.321077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.321792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.332150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.473741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 964/1942] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:28 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:28.478019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.486094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:28.494393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9523386-d1a5-4057-ad6b-90d50a6be3d9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.610360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.611217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.611724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.612120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.612785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.615120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.728248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 979/1943] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:28 2022] POST /identity/v3/users => generated 281 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:28.735044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.753770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27dcfa-c755-4cc4-a3b1-bb7b729c6a7e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.760573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.761213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.761483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.761931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.762782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.765363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.782309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 965/1944] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:28.788322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.809906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fadd79-1b30-45b4-b588-af47bd8ef0ab admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.819673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.820966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.821542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.822274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.823371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.826519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f31dbc-2120-42f8-9c8b-7a75b5e317e2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.849670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 980/1945] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:28.855639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:28.873197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be1c69-5f81-406e-9d51-37abc684f3f8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:28.881110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:28.885244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:28.885244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:28.885244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] PATH_INFO: `/v3/projects/cda3260057fb48788d4c176833e9cf2a/users/da04fc6b204040708ade254b69c609e4/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:28.912529 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda3260057fb48788d4c176833e9cf2a, user_id=da04fc6b204040708ade254b69c609e4, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:28.916721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:28.921813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:29.009124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 966/1946] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:28 2022] PUT /identity/v3/projects/cda3260057fb48788d4c176833e9cf2a/users/da04fc6b204040708ade254b69c609e4/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:29.024965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:29.026065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:29.026374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:29.059628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] MFA Rules not processed for user `da04fc6b204040708ade254b69c609e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:29.067557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:29.131340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 981/1947] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:29.168881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:29.170784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:29.246846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152714-3bf4-4d16-99db-a0097a6b3449 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:29.251178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:29.251794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:29.252078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:29.252357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:29.261035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:29.262521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065b159-3981-462c-9a86-7cd169722fdf service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:29.273345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 967/1948] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:29.290616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:29.367779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4161d7c-74f8-4618-a638-894c71c7bc08 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:29.373849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:29.375152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:29.375429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:29.376181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] PATH_INFO: `/v3/users/da04fc6b204040708ade254b69c609e4` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:29.378169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da04fc6b204040708ade254b69c609e4)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:29.384845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:29.401827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:29.662275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da04fc6b204040708ade254b69c609e4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:29.664747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da04fc6b204040708ade254b69c609e4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:29.820890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da04fc6b204040708ade254b69c609e4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:29.837250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da04fc6b204040708ade254b69c609e4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:29.864456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da04fc6b204040708ade254b69c609e4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:29.874179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 982/1949] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:29 2022] DELETE /identity/v3/users/da04fc6b204040708ade254b69c609e4 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:29.878914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:29.879552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:29.879755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:29.908837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:29.911428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:29.997298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 968/1950] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:30.013319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:30.015808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:30.092728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80dbeb-5310-4930-9aba-19e32b25c810 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:30.102264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:30.104057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.104057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.104525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.122997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:30.127428 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0394a2-1226-409b-a737-014b5f5ea0ab service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:30.150578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 983/1951] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:30.916807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:30.921555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:30.933881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0922c7-cc49-4802-9c35-ac4fe8a8a7b5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:30.939851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:30.940647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.940957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.941470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] PATH_INFO: `/v3/projects/cda3260057fb48788d4c176833e9cf2a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.942940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda3260057fb48788d4c176833e9cf2a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:30.945405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:31.119070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda3260057fb48788d4c176833e9cf2a', 'request_id': 'req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.125907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda3260057fb48788d4c176833e9cf2a', 'request_id': 'req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.254374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda3260057fb48788d4c176833e9cf2a', 'request_id': 'req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.290503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] The token cache is being invalidate because project cda3260057fb48788d4c176833e9cf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:31.291257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.292239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74c8ec3b-c3dc-41f7-854a-1a9b32bd6c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.301219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 969/1952] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:30 2022] DELETE /identity/v3/projects/cda3260057fb48788d4c176833e9cf2a => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:31.303804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.306610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.306610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.342456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:31.344334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.475196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 984/1953] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:31.478943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.480652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.481015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.524594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:31.524594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.542198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 970/1954] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:31.548045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:31.565828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5952-7306-41ed-8d9f-fff1fe0f03c5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:31.576496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:31.591051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.591533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.591926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.593356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:31.593723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:31.595840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1113c1-f3b3-4016-b26c-93f46ce8018d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:31.612282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 985/1955] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:31.617120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:31.628358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c787e8-eb37-45e6-ab33-0192bca8361f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:31.633868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:31.634441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.634764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.637671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.637671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:31.637865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:31.659372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:31.738921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.739194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.739570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.779787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 971/1956] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:31.781517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:31.823023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] MFA Rules not processed for user `f1381dae2b774b48a590e21b3d30dbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:31.825034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.850981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:31.925138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8febdea-0631-4fac-a87f-cb1307a37822 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:31.941503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:31.944485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.945486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.946221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.948697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:31.956354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.013961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 986/1957] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:32.019359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.021006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:32.037001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635bf6-d31f-470f-84fe-315c8fa11af5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.043674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.044388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.044744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.045090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.045871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.048252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.097922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 972/1958] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:32.102827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.113804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a9851-4a1d-42d9-a584-c7c18ade92a1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.118575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.119318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.119512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.119792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.120434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.122092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.135902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 973/1959] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:32.141510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.152162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e61a0b-a1d8-4abe-b523-8c34ab821d94 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.157272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.161659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.161659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.161659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.161659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.161659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.172920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 974/1960] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:32.177100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.190184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ca9d-ce1b-4374-8213-57f243c25a2e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.200950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.205439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 987/1961] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:32.205575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.205575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.205575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] PATH_INFO: `/v3/projects/d4bcc4f3881147f1beef02546b3f95e4/users/0a6b3424ea764f2e9260dfe1cca12a5d/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.207160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.228511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bcc4f3881147f1beef02546b3f95e4, user_id=0a6b3424ea764f2e9260dfe1cca12a5d, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.231487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab975-e8ad-490a-8f5d-0506b2f2934d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.294037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e338-7f72-4237-b13f-f495b9c67cc4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.299123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.299780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.300063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.300453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.301100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.303072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.366436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 975/1962] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:32 2022] PUT /identity/v3/projects/d4bcc4f3881147f1beef02546b3f95e4/users/0a6b3424ea764f2e9260dfe1cca12a5d/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:32.377919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.378210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.378596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.407923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 988/1963] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:32.412587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.412828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] MFA Rules not processed for user `0a6b3424ea764f2e9260dfe1cca12a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:32.414716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:32.432790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48692137-4058-4152-b15c-2c63d7914b76 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.504722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 976/1964] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:32.509487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.532763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.533722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.534206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.534828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.535715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.542303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.564950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 989/1965] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:32.567668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:32.575116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.601240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b237a-e64d-492e-994b-b407a5012c8f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.607800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.612497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.612497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.612497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] PATH_INFO: `/v3/projects/d579490e0cea488b97167ba13766fc47/users/4987e06b1e6744f384e42591dc8174bd/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.623669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2d463-cd8f-4bd7-9f3c-be6560fd402f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.630038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:32.634162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.634673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d579490e0cea488b97167ba13766fc47, user_id=4987e06b1e6744f384e42591dc8174bd, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.634980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.635250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.635535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.636201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.638218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.638369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.734725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 990/1966] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:32 2022] PUT /identity/v3/projects/d579490e0cea488b97167ba13766fc47/users/4987e06b1e6744f384e42591dc8174bd/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:32.739078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.777040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 977/1967] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/projects => generated 375 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:32.783368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.817234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2396-f7c4-4b8b-a476-94922635ebc9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.856053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74bcaa-009e-4f26-b6c7-ae8571ed6c7f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.863407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.865792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.865792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.865792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.865792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.873976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.887478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.888093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.888380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.888651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.889317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.890983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.901546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 991/1968] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:32.906718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.921961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dba3-0026-4c7a-8db4-6a8ccc4f5fea admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.929705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.930897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.931397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.931953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] PATH_INFO: `/v3/projects/d579490e0cea488b97167ba13766fc47/users/4987e06b1e6744f384e42591dc8174bd/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.935261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d579490e0cea488b97167ba13766fc47, user_id=4987e06b1e6744f384e42591dc8174bd, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.938238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.949786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 992/1969] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:32 2022] PUT /identity/v3/projects/d579490e0cea488b97167ba13766fc47/users/4987e06b1e6744f384e42591dc8174bd/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:32.960272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 978/1970] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:32.964273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.046507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c21436-162e-44f9-a630-459f52b661e6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.049653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.050339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.050628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.051015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.051731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.053577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.064869 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 993/1971] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:33.068949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.079825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf9fe-bec1-4b88-a797-ce799c1e363c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.085187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.088987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.088987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.088987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.088987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.088987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ace77-92ca-465b-a7b8-b91bb6066ee1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.101820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 979/1972] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:33.105931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.120402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463eb0-377f-46aa-b659-dd056e40d23d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.128615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.129384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.129669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.129946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] PATH_INFO: `/v3/projects/94c5e19175af45c282da10c7839f9d37/users/5f12d5fe72184245b1e1bfbd739e81a0/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.147570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:33.154302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c5e19175af45c282da10c7839f9d37, user_id=5f12d5fe72184245b1e1bfbd739e81a0, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.159239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.278840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 994/1973] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:33 2022] PUT /identity/v3/projects/94c5e19175af45c282da10c7839f9d37/users/5f12d5fe72184245b1e1bfbd739e81a0/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:33.294455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.294779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.295061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.323704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] MFA Rules not processed for user `5f12d5fe72184245b1e1bfbd739e81a0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:33.324951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:33.465581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 980/1974] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:33.473523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.598446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0957-db38-47ce-ab63-69cfe828cdc2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.601804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.602447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.602780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.605535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.605535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.607002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.730364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 995/1975] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:33 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:33.736842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.782708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d43e2-c0c6-4e77-a05e-bb98eab564f6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.823209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:33.895467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.896125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.896452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.896751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.897427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.899513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.039092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 981/1976] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:33 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:34.043940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:34.070060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422285db-4e92-47cc-b703-ebb30da09210 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:34.077630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:34.080451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.080451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.080451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.080451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:34.084326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.113235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 996/1977] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:34.117921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:34.129474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71373b-6e12-44d8-ab7c-d42df401e82b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:34.135051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:34.135776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.136064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.136460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] PATH_INFO: `/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5/users/d75c24274f7f4537943ba8916b119fa3/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.157681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb383b8bdfc4116940163fc01e7b5a5, user_id=d75c24274f7f4537943ba8916b119fa3, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:34.160449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.287006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 982/1978] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:34 2022] PUT /identity/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5/users/d75c24274f7f4537943ba8916b119fa3/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:34.291957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:34.306287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0a3ef-90a6-4b2a-915b-7660156888a4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:34.367598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:34.368253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.368532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.368805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.369475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:34.371141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00407-c57b-4c22-8ed4-36197aacf810 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.383369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 997/1979] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:34.389312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:34.402504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71cedc-8073-4f54-a4f2-ef236805e451 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:34.409406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:34.410249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.410531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.410840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] PATH_INFO: `/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5/users/d75c24274f7f4537943ba8916b119fa3/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.413677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb383b8bdfc4116940163fc01e7b5a5, user_id=d75c24274f7f4537943ba8916b119fa3, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:34.416961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.429436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 983/1980] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:34 2022] PUT /identity/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5/users/d75c24274f7f4537943ba8916b119fa3/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:34.442554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.442983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.443271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.479873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] MFA Rules not processed for user `d75c24274f7f4537943ba8916b119fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:34.481489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:34.552938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:34.592967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 998/1981] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:34.613637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:34.616063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:34.714139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb22ee1-c1c6-49a2-bef9-61e196ea6b56 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:34.718872 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:34.722426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:34.722426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:34.722426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:34.807431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:34.808974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:34.819754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 984/1982] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:38.062559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.064771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.084933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e11f-6beb-4baa-93c2-421dd77bb3d4 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.093246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.099352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.100921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.101662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.199809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.201659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea186ad-742e-4e0e-9a6c-01db87245915 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.224167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 999/1983] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:39.571449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.573667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:39.629333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:39.704055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c509cf-6f5f-4add-8680-763a4692635f service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.717692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.718461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.718877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.719258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] PATH_INFO: `/v3/users/d409a2de6e5a477f94a48d2830cb84d7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.721667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d409a2de6e5a477f94a48d2830cb84d7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.725811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.056914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd409a2de6e5a477f94a48d2830cb84d7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:40.161224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd409a2de6e5a477f94a48d2830cb84d7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:40.189380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd409a2de6e5a477f94a48d2830cb84d7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:40.190898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd409a2de6e5a477f94a48d2830cb84d7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:40.226412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd409a2de6e5a477f94a48d2830cb84d7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:40.240706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 985/1984] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:39 2022] DELETE /identity/v3/users/d409a2de6e5a477f94a48d2830cb84d7 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:40.243133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.243874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.243874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.293563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:40.296054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.501897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1000/1985] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:40.515210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.516927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.631659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a9f1-cda3-48e0-8d18-dd3ec3541046 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.651209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.656243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.657066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.657506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.658461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.659309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.684318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.687158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb6269-30df-40f5-8644-beb398a04cc6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.722860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 986/1986] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:40.786098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c223b5-f784-44be-ae04-17e8a8d2da67 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.793288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.795968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.797166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.798331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.802509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.860894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:40.897694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.899241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.916257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1001/1987] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:41.010816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.011203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.011545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.061390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] MFA Rules not processed for user `4987e06b1e6744f384e42591dc8174bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:41.063029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:41.122335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:41.174666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 987/1988] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:41.189308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.190893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:41.314114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f80c6-8240-449c-8d5d-b58a4fea93d8 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.320136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.320770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.321104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.321551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.325208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:41.328268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.329847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:41.363437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f6493-f3f5-429b-8d88-9d6adb6c0464 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.370263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.371048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.371399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.371742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] PATH_INFO: `/v3/projects/d274921882aa4770ac4eca4faeb8ad28` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.373963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d274921882aa4770ac4eca4faeb8ad28)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.377870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.423079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.424724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.434460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1002/1989] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:41.496649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd274921882aa4770ac4eca4faeb8ad28', 'request_id': 'req-617358c2-da62-4b95-8aae-818c3b885e6b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.503605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd274921882aa4770ac4eca4faeb8ad28', 'request_id': 'req-617358c2-da62-4b95-8aae-818c3b885e6b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.598259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd274921882aa4770ac4eca4faeb8ad28', 'request_id': 'req-617358c2-da62-4b95-8aae-818c3b885e6b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.614926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] The token cache is being invalidate because project d274921882aa4770ac4eca4faeb8ad28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:41.615238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.615984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.617734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 988/1990] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:41 2022] DELETE /identity/v3/projects/d274921882aa4770ac4eca4faeb8ad28 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:41.622069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.633596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72730d-e9cf-4f15-aac4-47077b3c1a2a service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.729392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.730489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.731034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.731567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] PATH_INFO: `/v3/users/b3d0aeebbe5740aa9040695be8f9c024` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.734288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d0aeebbe5740aa9040695be8f9c024)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.738316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.775738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:41.971511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d0aeebbe5740aa9040695be8f9c024'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:41.972271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d0aeebbe5740aa9040695be8f9c024'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:42.064441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d0aeebbe5740aa9040695be8f9c024'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:42.072918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d0aeebbe5740aa9040695be8f9c024'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:42.085628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38260ad8-52bd-4e45-8e61-548d0ff6666b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d0aeebbe5740aa9040695be8f9c024'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:42.097870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1003/1991] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:41 2022] DELETE /identity/v3/users/b3d0aeebbe5740aa9040695be8f9c024 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:42.727398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:42.729781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:42.746035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617358c2-da62-4b95-8aae-818c3b885e6b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:42.844406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:42.845062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:42.845341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:42.845744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] PATH_INFO: `/v3/projects/df97dc3995f44b8688f45b033cebf582` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:42.846987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df97dc3995f44b8688f45b033cebf582)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:42.848973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:43.012488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df97dc3995f44b8688f45b033cebf582', 'request_id': 'req-26b82f5a-c4b2-44e6-8c7e-c486679f400f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:43.018273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:43.024905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df97dc3995f44b8688f45b033cebf582', 'request_id': 'req-26b82f5a-c4b2-44e6-8c7e-c486679f400f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:43.144821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df97dc3995f44b8688f45b033cebf582', 'request_id': 'req-26b82f5a-c4b2-44e6-8c7e-c486679f400f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:43.188542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] The token cache is being invalidate because project df97dc3995f44b8688f45b033cebf582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:43.188673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:43.194466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-26b82f5a-c4b2-44e6-8c7e-c486679f400f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:43.194466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 989/1992] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:42 2022] DELETE /identity/v3/projects/df97dc3995f44b8688f45b033cebf582 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:43.199161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:43.199161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:43.199650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:43.258009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:43.259805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:43.440439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1004/1993] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:43.455906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:43.457367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:43.457958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:43.485400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:43.487013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:43.498703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 990/1994] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:43.500348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:43.511728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40ae1d-6020-4f19-ac45-af29a62f6200 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:43.517016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:43.517666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:43.518134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:43.518416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:43.519159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:43.519464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:43.521601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:43.545596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1005/1995] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:43.546405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:43.561773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40e1b-8730-4bd4-aff8-ffd945fb2b02 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:43.569618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:43.570274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:43.570558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:43.571117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:43.571671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:43.573787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:43.770752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 991/1996] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:43 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:43.774918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:43.789143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce735-5b21-44eb-8b0b-40276a56d094 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:43.865532 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:43.866258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:43.866543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:43.866841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:43.867593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:43.869679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:43.996861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1006/1997] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:43 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:44.001751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.015789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7a0ff-fcfe-44e4-8400-dec9627fbb33 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.022942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.023580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.024129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.024608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.025281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.027061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.041553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 992/1998] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:44.046828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.059603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb3a0f-0933-438b-b690-be6369f435de admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.066804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.067579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.067923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.068291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.069847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.071846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a1b2-12a5-420f-8bd9-7dc9d84abc86 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.076971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:44.094138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1007/1999] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:44.094520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.109588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a5c1-227b-4474-bfc7-077ed07c5d78 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.114732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.115513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.115896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.116235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] PATH_INFO: `/v3/projects/55db675a2a4f494a9ff8ff668d6fca93/users/b1d9aaec7f0242608b390ded889a3f9f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.132579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55db675a2a4f494a9ff8ff668d6fca93, user_id=b1d9aaec7f0242608b390ded889a3f9f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.135390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.255259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 993/2000] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:44 2022] PUT /identity/v3/projects/55db675a2a4f494a9ff8ff668d6fca93/users/b1d9aaec7f0242608b390ded889a3f9f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:44.267875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.268333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.268333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.313166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] MFA Rules not processed for user `b1d9aaec7f0242608b390ded889a3f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:44.317221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:44.419680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1008/2001] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:44.424208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.531292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341b63-ac48-492f-b049-e6c4eac6712d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.548945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.549584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.549867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.550614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.550830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.552943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.684496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 994/2002] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:44 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:44.690126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.712796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f6a7e-a8a0-44c4-9bb0-18deec840a61 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.803605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.804410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.804872 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.805196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.806103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.808362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.977152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.978347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1009/2003] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:44 2022] POST /identity/v3/users => generated 299 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:44.995254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433d413-50bf-435a-9e85-c7eb97089605 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:45.013422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:45.013422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.013422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.013422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.017163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:45.017163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:45.028234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 995/2004] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:45.036463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:45.049369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546f01-a41f-4561-9310-dcabbc843733 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:45.055129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:45.056083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.056350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.056629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] PATH_INFO: `/v3/projects/43558ae08c3a41ba925bba6d57abe237/users/9528e22a97d14256a72d6c688bcfe548/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.080504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43558ae08c3a41ba925bba6d57abe237, user_id=9528e22a97d14256a72d6c688bcfe548, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:45.081318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:45.189278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1010/2005] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:45 2022] PUT /identity/v3/projects/43558ae08c3a41ba925bba6d57abe237/users/9528e22a97d14256a72d6c688bcfe548/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:45.195083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:45.207301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef97d10-2879-4ca2-9b7f-98f8609adb5b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:45.285328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:45.292902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:45.293536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.293818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.294094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.295318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:45.297064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0007-4691-4671-9462-95c2d0561c48 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:45.312181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 996/2006] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:45.314242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:45.333192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209ce9-24f8-42a8-96db-1ee26fe2d4f9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:45.342591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:45.345160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.345448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.345770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] PATH_INFO: `/v3/projects/43558ae08c3a41ba925bba6d57abe237/users/9528e22a97d14256a72d6c688bcfe548/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.347880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43558ae08c3a41ba925bba6d57abe237, user_id=9528e22a97d14256a72d6c688bcfe548, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:45.350627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:45.368056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1011/2007] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:45 2022] PUT /identity/v3/projects/43558ae08c3a41ba925bba6d57abe237/users/9528e22a97d14256a72d6c688bcfe548/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:45.378644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.379006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.379285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.418990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] MFA Rules not processed for user `9528e22a97d14256a72d6c688bcfe548`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:45.421264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:45.550249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 997/2008] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:45.567818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:45.569661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:45.681881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92a9b2-e2c6-4c7a-807a-45e390922f09 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:45.686919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:45.687932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:45.688365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:45.688773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:45.703348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:45.705220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f2c15-c937-4d1b-97ca-edbcc323b5b3 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:45.717095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1012/2009] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:46.656458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:47.452312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:47.542555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56f365-42c9-4399-9da4-c6b9733e738d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:47.558690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:47.558690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:47.558690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:47.558690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] PATH_INFO: `/v3/users/b1d9aaec7f0242608b390ded889a3f9f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:47.559440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d9aaec7f0242608b390ded889a3f9f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:47.559440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:47.807222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d9aaec7f0242608b390ded889a3f9f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.907622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d9aaec7f0242608b390ded889a3f9f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.915333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d9aaec7f0242608b390ded889a3f9f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.916286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d9aaec7f0242608b390ded889a3f9f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.928399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:47.929588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d9aaec7f0242608b390ded889a3f9f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.935604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 998/2010] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:47 2022] DELETE /identity/v3/users/b1d9aaec7f0242608b390ded889a3f9f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:47.940109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:47.940574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:47.943746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:47.959431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:47.960599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:48.021053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1013/2011] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:48.031994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:48.034089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:48.105454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ac31-8ffc-4330-ad21-3b8f6e2256c6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:48.109915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:48.110568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:48.110901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:48.111176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:48.121732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:48.123285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:48.134292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 999/2012] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:48.684580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:48.686624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:48.699893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc22f34-ac07-47da-9d03-14e4d148eacf None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:48.705305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:48.705962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:48.706243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:48.706570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] PATH_INFO: `/v3/projects/55db675a2a4f494a9ff8ff668d6fca93` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:48.708016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55db675a2a4f494a9ff8ff668d6fca93)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:48.709940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:48.926203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55db675a2a4f494a9ff8ff668d6fca93', 'request_id': 'req-35aea77d-2ae1-4188-84d2-b441c430c7f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.933026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55db675a2a4f494a9ff8ff668d6fca93', 'request_id': 'req-35aea77d-2ae1-4188-84d2-b441c430c7f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.015915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55db675a2a4f494a9ff8ff668d6fca93', 'request_id': 'req-35aea77d-2ae1-4188-84d2-b441c430c7f1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.032613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] The token cache is being invalidate because project 55db675a2a4f494a9ff8ff668d6fca93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:49.032902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.033502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.040251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1014/2013] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:48 2022] DELETE /identity/v3/projects/55db675a2a4f494a9ff8ff668d6fca93 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:49.042489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:49.057367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c31e-6ff8-47fe-a5ae-9b2104fb39d5 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:49.123828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:49.124741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:49.125184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:49.125467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] PATH_INFO: `/v3/users/9528e22a97d14256a72d6c688bcfe548` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:49.130217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9528e22a97d14256a72d6c688bcfe548)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:49.130217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:49.373326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9528e22a97d14256a72d6c688bcfe548'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.465177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9528e22a97d14256a72d6c688bcfe548'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.480862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9528e22a97d14256a72d6c688bcfe548'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.482768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9528e22a97d14256a72d6c688bcfe548'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.513803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9528e22a97d14256a72d6c688bcfe548'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.524473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1000/2014] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:49 2022] DELETE /identity/v3/users/9528e22a97d14256a72d6c688bcfe548 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:49.961639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:49.965987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:49.987733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aea77d-2ae1-4188-84d2-b441c430c7f1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.033656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.036340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.075419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:50.089408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.090384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.090708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.090998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] PATH_INFO: `/v3/users/0a6b3424ea764f2e9260dfe1cca12a5d` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.092441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6b3424ea764f2e9260dfe1cca12a5d)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.094392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.101366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:50.135581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8d7a-4a27-4007-82f2-a38ecf9b2219 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.141456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.142164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.142713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.143148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] PATH_INFO: `/v3/projects/43558ae08c3a41ba925bba6d57abe237` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.144696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=43558ae08c3a41ba925bba6d57abe237)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.146755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.315364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6b3424ea764f2e9260dfe1cca12a5d'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.316305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6b3424ea764f2e9260dfe1cca12a5d'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.322982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43558ae08c3a41ba925bba6d57abe237', 'request_id': 'req-e4dad39d-8392-4025-9ad6-204abfbf2fad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.329566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43558ae08c3a41ba925bba6d57abe237', 'request_id': 'req-e4dad39d-8392-4025-9ad6-204abfbf2fad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.416680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6b3424ea764f2e9260dfe1cca12a5d'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.426818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6b3424ea764f2e9260dfe1cca12a5d'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.443678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6261c8f4-e633-43dd-832a-c7dddc95b2ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6b3424ea764f2e9260dfe1cca12a5d'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.451389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1015/2015] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:49 2022] DELETE /identity/v3/users/0a6b3424ea764f2e9260dfe1cca12a5d => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:50.456859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.457189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.457423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.477374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:50.478534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.532880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43558ae08c3a41ba925bba6d57abe237', 'request_id': 'req-e4dad39d-8392-4025-9ad6-204abfbf2fad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.548410 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] The token cache is being invalidate because project 43558ae08c3a41ba925bba6d57abe237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:50.548721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.549359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e4dad39d-8392-4025-9ad6-204abfbf2fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:50.555056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1001/2016] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:50 2022] DELETE /identity/v3/projects/43558ae08c3a41ba925bba6d57abe237 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:50.561973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a986138-6563-422e-a8d9-ac8fff6cbb60 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.561973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a986138-6563-422e-a8d9-ac8fff6cbb60 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.562206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1016/2017] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.562299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a986138-6563-422e-a8d9-ac8fff6cbb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.572079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.574397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.587834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1a986138-6563-422e-a8d9-ac8fff6cbb60 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:50.590124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a986138-6563-422e-a8d9-ac8fff6cbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.599119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1002/2018] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.607521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.607844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.608078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.633147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:50.634385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.655279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d25a5-37ac-4a36-8e95-284cb3d5c007 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.658823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1003/2019] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.662680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.663753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.664432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.664720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.665068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.673862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f31d0e-207c-4063-be67-14088ae2e51c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.677110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.678592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.681453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.682216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.682611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.683020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.684619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:50.685565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.687476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.692423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1017/2020] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:50.717287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1004/2021] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:50.719931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.730283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd1c3-e87d-4413-a708-0e4c319190c8 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.740785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.741557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.741682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.741958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.743340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.745501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.849408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1018/2022] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:50.853931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.865430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3bf78-ba91-4423-947b-7bdbb3e1e325 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.954425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.955097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.955577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.955877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.956526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.959037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.059193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1005/2023] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:51.065505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.082253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543d4f-fed4-4fed-919b-11692e83d6d4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.087400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.088024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.088309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.088586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.089325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.090986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.101927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:51.109718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1019/2024] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:51.112400 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.124842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb785-2435-4f42-8ad2-8f6fe709560d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.130100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.131052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.131549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.131711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.132356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.134095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.141617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:51.148432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1006/2025] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:51.158195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.169396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e42b-01a9-4e2b-8ea8-67d2ef4c12da admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.175108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.175954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.176341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.176688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] PATH_INFO: `/v3/projects/c8c572003fce48bd93985de7d3db28e5/users/0fde2607361241f6b8e5d25fb8abc92e/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.180413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.182526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:51.205221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c572003fce48bd93985de7d3db28e5, user_id=0fde2607361241f6b8e5d25fb8abc92e, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.209429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe4cb-20ab-492e-8d8d-ba310cff8e30 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.212319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb341c-91ad-42b5-92c8-bf28ab7e6c1f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.220955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.221626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.221946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.222176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] PATH_INFO: `/v3/projects/d4bcc4f3881147f1beef02546b3f95e4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.223462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4bcc4f3881147f1beef02546b3f95e4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.226054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.331894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1020/2026] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:51 2022] PUT /identity/v3/projects/c8c572003fce48bd93985de7d3db28e5/users/0fde2607361241f6b8e5d25fb8abc92e/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:51.353272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4bcc4f3881147f1beef02546b3f95e4', 'request_id': 'req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.353542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.353925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.354390 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.368850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4bcc4f3881147f1beef02546b3f95e4', 'request_id': 'req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.406565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] MFA Rules not processed for user `0fde2607361241f6b8e5d25fb8abc92e`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:51.407856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:51.497046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1021/2027] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:51.509675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.511399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:51.564187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4bcc4f3881147f1beef02546b3f95e4', 'request_id': 'req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.577395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] The token cache is being invalidate because project d4bcc4f3881147f1beef02546b3f95e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:51.577816 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.578498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.580388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1007/2028] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:51 2022] DELETE /identity/v3/projects/d4bcc4f3881147f1beef02546b3f95e4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:51.584055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:51.585546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:51.586592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcbba-ddec-418e-9481-8373e0bfe717 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.590640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.591909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.592213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.592578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.597244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:51.612096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.613502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.640691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1022/2029] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:51.688757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd61c8-7cc8-4a4d-84a4-4f9a90210db9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:51.694749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:51.695620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:51.697590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:51.697953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] PATH_INFO: `/v3/users/5f12d5fe72184245b1e1bfbd739e81a0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:51.699289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f12d5fe72184245b1e1bfbd739e81a0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:51.701360 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:51.897249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f12d5fe72184245b1e1bfbd739e81a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.964519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f12d5fe72184245b1e1bfbd739e81a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.978251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f12d5fe72184245b1e1bfbd739e81a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.979093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f12d5fe72184245b1e1bfbd739e81a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:51.994930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f12d5fe72184245b1e1bfbd739e81a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.002853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1008/2030] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:51 2022] DELETE /identity/v3/users/5f12d5fe72184245b1e1bfbd739e81a0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:52.656410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:52.671597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d57f-98a1-4afc-8959-7eb1d7fcfde2 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:52.791083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:52.792034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.792509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.792792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] PATH_INFO: `/v3/projects/94c5e19175af45c282da10c7839f9d37` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.794372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c5e19175af45c282da10c7839f9d37)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:52.797449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:52.805823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:53.015224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c5e19175af45c282da10c7839f9d37', 'request_id': 'req-3e8bd9f2-997d-4421-baa9-f9d36764f413'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.023878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c5e19175af45c282da10c7839f9d37', 'request_id': 'req-3e8bd9f2-997d-4421-baa9-f9d36764f413'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.153749 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c5e19175af45c282da10c7839f9d37', 'request_id': 'req-3e8bd9f2-997d-4421-baa9-f9d36764f413'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.171848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] The token cache is being invalidate because project 94c5e19175af45c282da10c7839f9d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:53.172225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.173366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.178138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1023/2031] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:52 2022] DELETE /identity/v3/projects/94c5e19175af45c282da10c7839f9d37 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:53.180152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.182356 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:53.192638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:53.198432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e4f5-ca30-4576-b848-f8a2d38a51fa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.267369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.268020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.268300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.268575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] PATH_INFO: `/v3/users/d75c24274f7f4537943ba8916b119fa3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.269729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75c24274f7f4537943ba8916b119fa3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.271732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.508154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75c24274f7f4537943ba8916b119fa3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.581264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75c24274f7f4537943ba8916b119fa3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.590057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75c24274f7f4537943ba8916b119fa3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.590939 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75c24274f7f4537943ba8916b119fa3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.603869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bee43fa-36f3-44ca-8050-2889979ff3e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75c24274f7f4537943ba8916b119fa3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.614062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1009/2032] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:53 2022] DELETE /identity/v3/users/d75c24274f7f4537943ba8916b119fa3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:54.152499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.154210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.172143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bd9f2-997d-4421-baa9-f9d36764f413 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.331210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.338607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.338607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.338930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] PATH_INFO: `/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.340303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb383b8bdfc4116940163fc01e7b5a5)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.342743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.551900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb383b8bdfc4116940163fc01e7b5a5', 'request_id': 'req-e115f687-99e1-42fa-8da5-bd6b4d8001b7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.557948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb383b8bdfc4116940163fc01e7b5a5', 'request_id': 'req-e115f687-99e1-42fa-8da5-bd6b4d8001b7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.703702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb383b8bdfc4116940163fc01e7b5a5', 'request_id': 'req-e115f687-99e1-42fa-8da5-bd6b4d8001b7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.719528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] The token cache is being invalidate because project bbb383b8bdfc4116940163fc01e7b5a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:54.719824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.720584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e115f687-99e1-42fa-8da5-bd6b4d8001b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.722071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1024/2033] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:54 2022] DELETE /identity/v3/projects/bbb383b8bdfc4116940163fc01e7b5a5 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:54.727488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.728186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.728186 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.753055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:54.755602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.829343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1010/2034] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:54.837299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.837765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.838133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.842394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.843846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.856193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b986c-d1f0-4d39-9c17-581f357498ff None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.859808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:54.861042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.861615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.862260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.862540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.862923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] PATH_INFO: `/v3/users/167b45b560364793ba924fc2bbeeafd5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.864099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167b45b560364793ba924fc2bbeeafd5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.866066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.869568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1025/2035] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:54.873857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.883421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe6a0d-66b5-435e-9f4d-a22eacea9404 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.884060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:54.892094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.893004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.893356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.893681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.894475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:54.894923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.900777 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.910358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1026/2036] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:54.915193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.921647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:54.925370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fac5-1c02-4dc2-9dc6-e3a1c19cba50 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.930774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.931379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.931665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.931943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.932564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.934606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.080689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1027/2037] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:54 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:55.086831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.133734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167b45b560364793ba924fc2bbeeafd5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.187692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383badf-db2b-4791-9457-beb4563acfc4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.194779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.195408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.195687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.195967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.196599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.198846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.229372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167b45b560364793ba924fc2bbeeafd5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.239224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167b45b560364793ba924fc2bbeeafd5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.240313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167b45b560364793ba924fc2bbeeafd5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.265265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-874bdcfa-e9bb-4945-96aa-533d843d4599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167b45b560364793ba924fc2bbeeafd5'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.276239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1011/2038] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:54 2022] DELETE /identity/v3/users/167b45b560364793ba924fc2bbeeafd5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:55.297761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.300595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.300595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.308361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1028/2039] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:55 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:55.310453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.318388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.339334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ce9d-265b-404f-ad6e-a9ce6fb2ee58 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.345201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:55.345475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.457575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.458207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.458490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.458849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.459503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.462251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.475135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1012/2040] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:55.485239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1029/2041] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:55.485600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.487891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.489380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.501096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6795a-e1c7-4e05-b199-059f1efa7778 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.509150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.509796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.510200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.511615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.511615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.513113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.547911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1030/2042] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:55.552817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.571128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91a89d-a364-41a5-afad-e82081c2c4b8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.579016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.580344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.580641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.581122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] PATH_INFO: `/v3/projects/fadf4161dc28487487541cefeae7a220/users/f7f0342a701f492a9de96588c1569fac/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.584849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46654704-1aa5-4e78-a1fc-9418087ce970 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.589661 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.597860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.597860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.597860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.603137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.604634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecdcb8-5284-4758-a027-c6095cd5b2e4 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.608483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fadf4161dc28487487541cefeae7a220, user_id=f7f0342a701f492a9de96588c1569fac, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.622133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.623064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1013/2043] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:55.767175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1031/2044] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:55 2022] PUT /identity/v3/projects/fadf4161dc28487487541cefeae7a220/users/f7f0342a701f492a9de96588c1569fac/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:55.769436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.769879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.770223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.805234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:55.807704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.841691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:55.878568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1014/2045] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:55.878994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.927490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:55.984799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802637b-be14-43ef-936b-cb4b10ed466d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.991575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.992622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.993473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.993929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.013948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.016947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.032954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1032/2046] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:56.245143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.245143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:56.259069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8459-3294-4e58-9cdb-6019a4c8136f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.264448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.265308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.265964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.266392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] PATH_INFO: `/v3/projects/0767684cf0e64d1aa0a337d55a705433` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.267942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0767684cf0e64d1aa0a337d55a705433)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.270099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.411112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0767684cf0e64d1aa0a337d55a705433', 'request_id': 'req-d8d76334-4e97-401c-b8a2-ac218a1d190f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.418128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0767684cf0e64d1aa0a337d55a705433', 'request_id': 'req-d8d76334-4e97-401c-b8a2-ac218a1d190f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.495626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0767684cf0e64d1aa0a337d55a705433', 'request_id': 'req-d8d76334-4e97-401c-b8a2-ac218a1d190f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.509050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] The token cache is being invalidate because project 0767684cf0e64d1aa0a337d55a705433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:56.509341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.510105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.511681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1015/2047] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:56 2022] DELETE /identity/v3/projects/0767684cf0e64d1aa0a337d55a705433 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:56.515902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.527029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6d1e-36c5-49e0-ab4d-1c01b3ccb498 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.603321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.603909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.604195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.604473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] PATH_INFO: `/v3/users/a250cbd0e72a487f87166ed7bf6d272c` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.605694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a250cbd0e72a487f87166ed7bf6d272c)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.608260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.942636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a250cbd0e72a487f87166ed7bf6d272c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:56.943421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a250cbd0e72a487f87166ed7bf6d272c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.010478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:57.012745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:57.037058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d76334-4e97-401c-b8a2-ac218a1d190f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:57.049459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:57.050540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.051061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.052040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] PATH_INFO: `/v3/users/0fde2607361241f6b8e5d25fb8abc92e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.055125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fde2607361241f6b8e5d25fb8abc92e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:57.058313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:57.075065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a250cbd0e72a487f87166ed7bf6d272c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.085623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a250cbd0e72a487f87166ed7bf6d272c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.117634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-19a68022-121b-4a64-aee3-b8bf03c4871b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a250cbd0e72a487f87166ed7bf6d272c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.126041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1033/2048] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:56 2022] DELETE /identity/v3/users/a250cbd0e72a487f87166ed7bf6d272c => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:57.405203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fde2607361241f6b8e5d25fb8abc92e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.598081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fde2607361241f6b8e5d25fb8abc92e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.612794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fde2607361241f6b8e5d25fb8abc92e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.614209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fde2607361241f6b8e5d25fb8abc92e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.631075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fde2607361241f6b8e5d25fb8abc92e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.639097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1016/2049] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:57 2022] DELETE /identity/v3/users/0fde2607361241f6b8e5d25fb8abc92e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:57.644026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.644529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.644655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.670886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:57.672468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:57.729171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:57.740443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1034/2050] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:57.751747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:57.753444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:57.762629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:57.766203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:57.786442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9ea3-2794-4bf9-a848-135402948a31 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:57.792356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:57.794493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.794493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.794857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] PATH_INFO: `/v3/projects/cdbb1aa9977147919cc5fbb1282fb808` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.796194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbb1aa9977147919cc5fbb1282fb808)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:57.798316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:57.823100 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:57.854870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3c725-8c4a-424f-a1cb-88a0b61a934e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:57.858965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:57.859588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.859861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.860128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.870567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:57.872654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a63026-712e-4f02-800e-c84547a0bfdc service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:57.882744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1017/2051] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:57.938343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbb1aa9977147919cc5fbb1282fb808', 'request_id': 'req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.953312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbb1aa9977147919cc5fbb1282fb808', 'request_id': 'req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.023587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbb1aa9977147919cc5fbb1282fb808', 'request_id': 'req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.037740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] The token cache is being invalidate because project cdbb1aa9977147919cc5fbb1282fb808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:58.038177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.038987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-6ec0a9d9-b87a-4954-a2b9-da9a55476b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.047270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1035/2052] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:57 2022] DELETE /identity/v3/projects/cdbb1aa9977147919cc5fbb1282fb808 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:58.052663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796231-94d1-4033-9701-368d70557f97 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.053551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796231-94d1-4033-9701-368d70557f97 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.053869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796231-94d1-4033-9701-368d70557f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.075596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0f796231-94d1-4033-9701-368d70557f97 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:58.076834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0f796231-94d1-4033-9701-368d70557f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.154988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1018/2053] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:58.163307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.163307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.163625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.190362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:58.192002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.205969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1036/2054] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:58.210090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796231-94d1-4033-9701-368d70557f97 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.230078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796231-94d1-4033-9701-368d70557f97 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.251611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.252432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.252819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.253499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.254322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:58.258888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.260893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01563-5561-4a5b-aba1-8f00cd21e227 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.292634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1019/2055] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:58.294384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.305730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d2c92-6bc7-45d8-84ae-af8c97ab6c43 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.318215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.318942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.319230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.319505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.320325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.322405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.422708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.423006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.423287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.474437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] MFA Rules not processed for user `f7f0342a701f492a9de96588c1569fac`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:58.475905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.562741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1020/2056] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:58.585019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.589590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.663271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d5a46-10da-4ef1-8c87-55c5cd1f67cf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.667472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.668099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.668381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.668716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.679184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.686479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.690623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1021/2057] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:58.720683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1037/2058] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:58 2022] POST /identity/v3/projects => generated 367 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:58.723477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.744357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b5f9-5c4b-4ad1-a8d7-3939898443d9 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.763429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.769863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.892285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.896029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.896643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.898082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.898217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd918d28-15b8-4a70-a09c-daebcc75df82 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.899649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.904749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.906114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.907452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.908036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.908510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] PATH_INFO: `/v3/projects/c8c572003fce48bd93985de7d3db28e5` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.910041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c572003fce48bd93985de7d3db28e5)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.912632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.023417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1022/2059] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:58 2022] POST /identity/v3/users => generated 284 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:59.029396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.045589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea13d87-d5e1-403a-aa78-247d09bb67ac admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.050966 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.051591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.051913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.052185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.052848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.054936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.079967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1023/2060] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:59.086575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.092011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c572003fce48bd93985de7d3db28e5', 'request_id': 'req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.099021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c572003fce48bd93985de7d3db28e5', 'request_id': 'req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.105211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aede9-7bc5-4f0f-a540-1165f7f21777 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.182297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.182983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.183266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.183546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.184211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.185982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.201970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1024/2061] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:59.209376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.221868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cbb5a-ce1f-4683-8ebf-b0efaae2d2aa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.227409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c572003fce48bd93985de7d3db28e5', 'request_id': 'req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.231084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.233254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.233254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.233254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] PATH_INFO: `/v3/projects/2af55b0fcb564260ade41958d02abd6a/users/dad9af3d7e2c4877aa83cd294a8ee1a9/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.273774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] The token cache is being invalidate because project c8c572003fce48bd93985de7d3db28e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:59.274037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.274201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af55b0fcb564260ade41958d02abd6a, user_id=dad9af3d7e2c4877aa83cd294a8ee1a9, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.275023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2defba2d-5c96-44cf-b3f8-6a54bb2cb240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.276773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1038/2062] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:58 2022] DELETE /identity/v3/projects/c8c572003fce48bd93985de7d3db28e5 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:59.280745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.285863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6f4e-7cc0-4b61-8c27-965dea9eec6d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.289442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6f4e-7cc0-4b61-8c27-965dea9eec6d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.289442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6f4e-7cc0-4b61-8c27-965dea9eec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.326680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7fdb6f4e-7cc0-4b61-8c27-965dea9eec6d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.327855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7fdb6f4e-7cc0-4b61-8c27-965dea9eec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.337293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1039/2063] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:59.345005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.345323 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.345596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.374483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.375932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.384505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1040/2064] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:59.389491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.402065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52016cc-41cf-4964-8770-e716bce004b9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.410508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.411275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.411561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.412183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.412570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:59.412972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.414838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c81fe2-c701-4f35-ab93-b90317fcc196 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.442677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1041/2065] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:59.445318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1025/2066] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:59 2022] PUT /identity/v3/projects/2af55b0fcb564260ade41958d02abd6a/users/dad9af3d7e2c4877aa83cd294a8ee1a9/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:59.458297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.460504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.460504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.460727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.502772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] MFA Rules not processed for user `dad9af3d7e2c4877aa83cd294a8ee1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.504245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.618205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7629db7-11ff-4719-b5da-242bcf92e1c1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.631377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1042/2067] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:59.670693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.670693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.670693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.670693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.671369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.671369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.677042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.682610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.726701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01881ff-faf4-4202-876a-24912fca1d37 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.756782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.757226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.757585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.758058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.759277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.764719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.834271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1026/2068] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/projects => generated 363 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:59.839604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.933443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1043/2069] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/projects => generated 367 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:59.935799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.956191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:59.985084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c0e9f-707e-4151-8dd8-ba981e76c143 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.071676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf695c4-e05f-4ce1-a1b4-0b30a7bd21e8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.082634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.083380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.083635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.083967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.084721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.088848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.129818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.130706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.131095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.131482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.132233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.134630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.225736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1027/2070] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/users => generated 282 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:00.229669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1044/2071] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:00.230928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.234228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.243205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f945d-bad6-47e2-a43d-037c246d9d1b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.247601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40edcfa-2fd0-4bdf-b52f-ec3e2c47a068 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.249839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.250938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.251451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.251890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.252732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.253822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.254614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.254968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.255375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.255500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.257500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.260481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.277635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1045/2072] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.279413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1028/2073] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.281535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.284267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.293154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:00.299619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114547-6a9b-4e83-9a33-3888c53abacd admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.300334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf721-e84c-46b1-91a6-7a8901f1b495 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.307898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.308224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.308712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.309225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.309431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.310110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] PATH_INFO: `/v3/projects/90e42435d75742019aaff626dac68768/users/0319b2d8b0444161884c66139c070a03/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.310509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.310972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.311804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.316931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.336133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1046/2074] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.339432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e42435d75742019aaff626dac68768, user_id=0319b2d8b0444161884c66139c070a03, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.339598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.339703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.352632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2e00-3a31-429a-85f4-00cd339cc4ad admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.361255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.368967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.368967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.368967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] PATH_INFO: `/v3/projects/db4ed6100ce34e16bf75d9e8f5e5092d/users/6de08e32cefe42ddb5c05bf02778eeea/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.384739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4ed6100ce34e16bf75d9e8f5e5092d, user_id=6de08e32cefe42ddb5c05bf02778eeea, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.388042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1c14a-f450-45d1-9cf6-c1af3b78ac3c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.430714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1029/2075] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:00 2022] PUT /identity/v3/projects/90e42435d75742019aaff626dac68768/users/0319b2d8b0444161884c66139c070a03/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:00.436270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.493298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1047/2076] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:00 2022] PUT /identity/v3/projects/db4ed6100ce34e16bf75d9e8f5e5092d/users/6de08e32cefe42ddb5c05bf02778eeea/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:00.506891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.507251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.507535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.560623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ef9c-e12f-4471-be97-0cfbc4ea1f8d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.561780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] MFA Rules not processed for user `6de08e32cefe42ddb5c05bf02778eeea`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.565108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.675955 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.676898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.677192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.677442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.678123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.679957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.693594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1030/2077] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.698960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.712275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d588a-10f1-4bce-97b1-563d53953096 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.722012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.722554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.723092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.723628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] PATH_INFO: `/v3/projects/90e42435d75742019aaff626dac68768/users/0319b2d8b0444161884c66139c070a03/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.726226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e42435d75742019aaff626dac68768, user_id=0319b2d8b0444161884c66139c070a03, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.729048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f511c-78aa-4aeb-bbd3-a9186170c415 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.736048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1048/2078] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:00.743974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1031/2079] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:00 2022] PUT /identity/v3/projects/90e42435d75742019aaff626dac68768/users/0319b2d8b0444161884c66139c070a03/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:00.747910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.749367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.754332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280205e-1453-4b58-9d4d-6596c8550855 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.758474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280205e-1453-4b58-9d4d-6596c8550855 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.758474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280205e-1453-4b58-9d4d-6596c8550855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.794887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d280205e-1453-4b58-9d4d-6596c8550855 None None] MFA Rules not processed for user `0319b2d8b0444161884c66139c070a03`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.804893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d280205e-1453-4b58-9d4d-6596c8550855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.825061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909bdb6-2476-4a63-8fdb-81d9b7be8e84 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.901606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1032/2080] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:00.904560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.904852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.905157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.923185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.924251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.924730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.925200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.945803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] MFA Rules not processed for user `f7f0342a701f492a9de96588c1569fac`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.948202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.036537 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1033/2081] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:01.037779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.039353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.053507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb361ac-6995-4ea7-a1ff-5a86c34ff74f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.058515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.059654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.060111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.061183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.065667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.068528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bfa1e-289c-4d66-a88d-4cea1409f328 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.073902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.076106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.088945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1034/2082] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:01.114553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1049/2083] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:01.119735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:01.244006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.258037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78857cf-2b71-41da-b335-54ad08c93dca service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.263626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.264026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.264522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.265131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.358341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.360051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0483f-b57d-48a2-b694-918d74d940d0 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.371222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1035/2084] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:01.910462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:02.170946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:02.919518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.920180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.920180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.921259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1050/2085] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:58:02.933506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.934334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.934334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.975203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] MFA Rules not processed for user `440669b078bd474d955d2faea0dbe859`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:02.977373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:03.061853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1036/2086] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:03.074289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.076481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:03.096812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff97784-0cfb-4985-afae-0639fc6f95c0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.102955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.103746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.104175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.104554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.114862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.116527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.129914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1051/2087] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:03.145490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.147384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:03.250179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363c8f5-dc61-47d2-992d-cafa1f941f49 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.250179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.250179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.250482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.250877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] PATH_INFO: `/v3/users/0737c1b1280c468fa9df3ba7bc9ee6b6` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.252899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0737c1b1280c468fa9df3ba7bc9ee6b6)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.255153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.629212 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0737c1b1280c468fa9df3ba7bc9ee6b6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:03.761534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0737c1b1280c468fa9df3ba7bc9ee6b6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:03.774861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0737c1b1280c468fa9df3ba7bc9ee6b6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:03.776284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0737c1b1280c468fa9df3ba7bc9ee6b6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:03.812897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-59ddb9e9-e0fa-4335-88e8-6bd41d92471b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0737c1b1280c468fa9df3ba7bc9ee6b6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:03.819754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1037/2088] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:03 2022] DELETE /identity/v3/users/0737c1b1280c468fa9df3ba7bc9ee6b6 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:03.835896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:04.133467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.165983 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d67f-1f4f-40fc-a8bc-2589302a1dba service placement] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.276856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.277831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.278708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.279069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] PATH_INFO: `/v3/projects/f58ac192b94c47179055f35cae59f8ec` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.280295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58ac192b94c47179055f35cae59f8ec)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.282387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:04.423135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58ac192b94c47179055f35cae59f8ec', 'request_id': 'req-b2f1a939-ff0f-4552-9384-a503471cf6f0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.429703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58ac192b94c47179055f35cae59f8ec', 'request_id': 'req-b2f1a939-ff0f-4552-9384-a503471cf6f0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.490100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:04.562714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58ac192b94c47179055f35cae59f8ec', 'request_id': 'req-b2f1a939-ff0f-4552-9384-a503471cf6f0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.567781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] The token cache is being invalidate because project f58ac192b94c47179055f35cae59f8ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:04.568072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.568727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b2f1a939-ff0f-4552-9384-a503471cf6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.570297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1052/2089] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:04 2022] DELETE /identity/v3/projects/f58ac192b94c47179055f35cae59f8ec => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:04.581609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.582099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.582202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.604878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:04.606108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:04.670034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1038/2090] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:04.679955 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.680246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.681352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.702160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:04.703421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:04.711647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1053/2091] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:04.716185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.728958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a38126-85cf-44c6-9afa-105b389f3102 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.731590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.732242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.732518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.732791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.733439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:04.733737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.735369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:04.747524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1039/2092] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:04.752108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.762079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa36aaa-ca79-48f3-b855-fe5b8daaf37f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.768457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.769119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.769799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.770148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.770930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.773155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:04.874622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1054/2093] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:04 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:04.878864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.890152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f459cb8-6c9e-46a6-8221-56886abde556 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.949358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.949992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.950351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.950641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.953327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.955419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.072236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1040/2094] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:04 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:05.076765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.087202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5bef9-e9b0-499d-a67a-c4b1b02c7974 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.092431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.093032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.095975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.095975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.095975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.095975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.106627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1055/2095] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:05.110823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.121029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be470-5d15-4106-8fa7-f5f758350b67 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.128786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.128786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.128786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.128786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.129145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.130562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc7c4-d2af-42ae-bbda-ceae96b01f55 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.146968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1041/2096] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:05.148473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.159639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fc57-85b8-4341-b8b3-024741e6a936 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.164607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.165540 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.165828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.166107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] PATH_INFO: `/v3/projects/14568fccd1ef4572a7cb33e4869a2c3e/users/1b353e743b4c414b9b8bcf7d830512b7/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.182989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14568fccd1ef4572a7cb33e4869a2c3e, user_id=1b353e743b4c414b9b8bcf7d830512b7, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.185875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.294598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1056/2097] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:05 2022] PUT /identity/v3/projects/14568fccd1ef4572a7cb33e4869a2c3e/users/1b353e743b4c414b9b8bcf7d830512b7/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:05.307057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.307554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.307747 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.347439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] MFA Rules not processed for user `1b353e743b4c414b9b8bcf7d830512b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:05.347439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:05.452973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1042/2098] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:05.456318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.605125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aed73c-a809-42a8-aafa-1ccebd388f11 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.617234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:05.622629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.623862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.624661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.625250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.627564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.632410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.808307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1057/2099] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:05 2022] POST /identity/v3/projects => generated 373 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:05.814971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.829792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f14d-ff5e-41b3-8a2a-0c6fbd218976 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.904023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.904659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.904938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.905211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.905873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.912572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.089124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1043/2100] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:05 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:06.091243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.104750 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0cbbb-cb39-4039-8112-0426c8cc148b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.110779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.111677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.112338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.112541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.113315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.115262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.128198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1058/2101] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:06.134761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.146446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34afafe-bdcb-4b01-b0b4-e39ceac589a3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.151895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.152648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.152985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.153352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] PATH_INFO: `/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4/users/496368d0f2c8419f8b5a3e70131d9a1b/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.181048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d94bc80cb4f4fb4f1dd29be514ae4, user_id=496368d0f2c8419f8b5a3e70131d9a1b, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.187433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.235565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:06.299439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1044/2102] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:06 2022] PUT /identity/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4/users/496368d0f2c8419f8b5a3e70131d9a1b/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:06.304613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.317176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb92c-b801-45c6-bb9f-97125b096dd9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.387261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.387885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.388173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.388452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.389158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.391050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418d3e4-151e-4264-8cb4-61470dc5a23a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.405481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1059/2103] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:06.408394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.433863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363174b-07c0-4a4d-a6b7-f26cf0787fc7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.433863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.433863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.433863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.434780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] PATH_INFO: `/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4/users/496368d0f2c8419f8b5a3e70131d9a1b/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.434780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d94bc80cb4f4fb4f1dd29be514ae4, user_id=496368d0f2c8419f8b5a3e70131d9a1b, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.434780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e323e-acdc-4fb1-b064-a7d11e188ca6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.446774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1045/2104] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:06 2022] PUT /identity/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4/users/496368d0f2c8419f8b5a3e70131d9a1b/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:06.458517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.458837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.459298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.493459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] MFA Rules not processed for user `496368d0f2c8419f8b5a3e70131d9a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:06.494744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.500683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.501002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.501222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.530183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] MFA Rules not processed for user `f1381dae2b774b48a590e21b3d30dbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:06.531726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.579553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1060/2105] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:06.591804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.593257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.593740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1046/2106] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:06.653742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95d43-4d9b-487d-a57c-77f11b042231 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.657826 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.658448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.658915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.659256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.671399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.672894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.683585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1061/2107] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:07.085388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:07.977956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:08.669876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:08.685913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:08.687379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:08.759827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fe49b-8221-48e3-bed8-3bdbc4279cea None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:08.767548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:08.768685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:08.771162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:08.771162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:08.771396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] PATH_INFO: `/v3/users/1b353e743b4c414b9b8bcf7d830512b7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:08.772685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b353e743b4c414b9b8bcf7d830512b7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:08.775125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:08.777729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b9a9-b4b0-49ad-a51b-afec3989026e service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:08.783267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:08.785891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:08.785891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:08.785891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] PATH_INFO: `/v3/users/dad9af3d7e2c4877aa83cd294a8ee1a9` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:08.786451 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad9af3d7e2c4877aa83cd294a8ee1a9)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:08.789593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:08.987042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b353e743b4c414b9b8bcf7d830512b7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.088562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad9af3d7e2c4877aa83cd294a8ee1a9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.089212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad9af3d7e2c4877aa83cd294a8ee1a9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.103338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b353e743b4c414b9b8bcf7d830512b7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.116119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b353e743b4c414b9b8bcf7d830512b7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.117433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b353e743b4c414b9b8bcf7d830512b7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.137634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c0fa2a15-8011-483a-bf91-68d9252ce808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b353e743b4c414b9b8bcf7d830512b7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.145345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1047/2108] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:08 2022] DELETE /identity/v3/users/1b353e743b4c414b9b8bcf7d830512b7 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:09.149766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66499aa-730e-46bb-acdd-439356543357 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:09.150056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66499aa-730e-46bb-acdd-439356543357 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:09.150365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66499aa-730e-46bb-acdd-439356543357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:09.183036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e66499aa-730e-46bb-acdd-439356543357 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:09.187639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e66499aa-730e-46bb-acdd-439356543357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:09.212237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad9af3d7e2c4877aa83cd294a8ee1a9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.221534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad9af3d7e2c4877aa83cd294a8ee1a9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.236765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:09.238592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a1ef530d-ebe6-4a19-9ebf-96638ec17268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad9af3d7e2c4877aa83cd294a8ee1a9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:09.250144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1062/2109] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:08 2022] DELETE /identity/v3/users/dad9af3d7e2c4877aa83cd294a8ee1a9 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:09.254472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:09.254979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:09.255411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:09.274857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1048/2110] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:09.278286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:09.280468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:09.283634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66499aa-730e-46bb-acdd-439356543357 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:09.285380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e66499aa-730e-46bb-acdd-439356543357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:09.369461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66499aa-730e-46bb-acdd-439356543357 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:09.369766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1063/2111] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:09.376998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:09.436843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:09.437462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:09.437739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:09.438080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:09.449954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:09.452097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:09.463678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1049/2112] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:09.482294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd259f-6398-43e3-86a1-c4c796b304c2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:09.485988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:09.486606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:09.488442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:09.488442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:09.503340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:09.505088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:09.520513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1064/2113] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:09.806560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:10.304503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:10.306062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:10.319850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cc7b4-06a9-4dd6-83d7-8d69765d0312 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:10.325250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:10.325884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:10.326166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:10.326439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] PATH_INFO: `/v3/projects/14568fccd1ef4572a7cb33e4869a2c3e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:10.327664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14568fccd1ef4572a7cb33e4869a2c3e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:10.331623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:10.337232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:10.338860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:10.352946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b45d86-eb6d-4fca-9f17-6b1d231e06ad service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:10.357979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:10.358624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:10.358948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:10.359226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] PATH_INFO: `/v3/projects/2af55b0fcb564260ade41958d02abd6a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:10.360372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af55b0fcb564260ade41958d02abd6a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:10.362397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:10.473644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14568fccd1ef4572a7cb33e4869a2c3e', 'request_id': 'req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.479792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14568fccd1ef4572a7cb33e4869a2c3e', 'request_id': 'req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.509387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af55b0fcb564260ade41958d02abd6a', 'request_id': 'req-151e98fa-5291-4740-9c25-254711d01ed7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.515695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af55b0fcb564260ade41958d02abd6a', 'request_id': 'req-151e98fa-5291-4740-9c25-254711d01ed7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.600388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14568fccd1ef4572a7cb33e4869a2c3e', 'request_id': 'req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.614124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] The token cache is being invalidate because project 14568fccd1ef4572a7cb33e4869a2c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:10.614495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.615182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.617235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1050/2114] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:10 2022] DELETE /identity/v3/projects/14568fccd1ef4572a7cb33e4869a2c3e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:10.620981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:10.623072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:10.669370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af55b0fcb564260ade41958d02abd6a', 'request_id': 'req-151e98fa-5291-4740-9c25-254711d01ed7'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.693464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] The token cache is being invalidate because project 2af55b0fcb564260ade41958d02abd6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:10.693464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.693464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:10.693464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1065/2115] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:10 2022] DELETE /identity/v3/projects/2af55b0fcb564260ade41958d02abd6a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:10.693464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:10.694882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:10.695993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2f62-df3e-4211-b0f4-fb8b37e75f04 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:10.757624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:10.758250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:10.758555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:10.758871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] PATH_INFO: `/v3/users/496368d0f2c8419f8b5a3e70131d9a1b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:10.760004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=496368d0f2c8419f8b5a3e70131d9a1b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:10.762505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:10.774978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e98fa-5291-4740-9c25-254711d01ed7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:10.781645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:10.782300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:10.782587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:10.782902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] PATH_INFO: `/v3/users/0319b2d8b0444161884c66139c070a03` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:10.784052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0319b2d8b0444161884c66139c070a03)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:10.785989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:10.909174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:11.018270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '496368d0f2c8419f8b5a3e70131d9a1b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.020094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0319b2d8b0444161884c66139c070a03'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.020945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0319b2d8b0444161884c66139c070a03'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.138064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '496368d0f2c8419f8b5a3e70131d9a1b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.145608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '496368d0f2c8419f8b5a3e70131d9a1b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.146324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '496368d0f2c8419f8b5a3e70131d9a1b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.159686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '496368d0f2c8419f8b5a3e70131d9a1b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.167140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1051/2116] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:10 2022] DELETE /identity/v3/users/496368d0f2c8419f8b5a3e70131d9a1b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:11.246592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0319b2d8b0444161884c66139c070a03'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.254814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0319b2d8b0444161884c66139c070a03'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.270487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0319b2d8b0444161884c66139c070a03'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:11.280619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1066/2117] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:10 2022] DELETE /identity/v3/users/0319b2d8b0444161884c66139c070a03 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:11.728922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:11.728922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:11.815658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162122e3-2387-406f-afbb-0b3ed4cbe7c7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:11.863866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:11.865404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:11.874030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:11.874717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:11.875005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:11.875281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] PATH_INFO: `/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:11.876472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5d94bc80cb4f4fb4f1dd29be514ae4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:11.878530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:11.935867 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f90fe0-1174-49ee-8344-2c3a62f1b706 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:11.940449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:11.941092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:11.941470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:11.941718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] PATH_INFO: `/v3/projects/90e42435d75742019aaff626dac68768` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:11.942897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e42435d75742019aaff626dac68768)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:11.952937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.034705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5d94bc80cb4f4fb4f1dd29be514ae4', 'request_id': 'req-93a63b7b-7a42-4b09-94f9-2049f283614a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.041627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5d94bc80cb4f4fb4f1dd29be514ae4', 'request_id': 'req-93a63b7b-7a42-4b09-94f9-2049f283614a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.102724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e42435d75742019aaff626dac68768', 'request_id': 'req-a02e6359-a9b2-49a4-9482-b480f30e2e9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.108479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e42435d75742019aaff626dac68768', 'request_id': 'req-a02e6359-a9b2-49a4-9482-b480f30e2e9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.112536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5d94bc80cb4f4fb4f1dd29be514ae4', 'request_id': 'req-93a63b7b-7a42-4b09-94f9-2049f283614a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.121609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:12.134122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] The token cache is being invalidate because project 0d5d94bc80cb4f4fb4f1dd29be514ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:12.134508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.135313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-93a63b7b-7a42-4b09-94f9-2049f283614a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.137643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1052/2118] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:11 2022] DELETE /identity/v3/projects/0d5d94bc80cb4f4fb4f1dd29be514ae4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:12.142927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dc2f-4ebe-4719-adb1-c94865ffc6f4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.143170 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dc2f-4ebe-4719-adb1-c94865ffc6f4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.143449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dc2f-4ebe-4719-adb1-c94865ffc6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.163925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5720dc2f-4ebe-4719-adb1-c94865ffc6f4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.165423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5720dc2f-4ebe-4719-adb1-c94865ffc6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:12.209467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e42435d75742019aaff626dac68768', 'request_id': 'req-a02e6359-a9b2-49a4-9482-b480f30e2e9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.223142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] The token cache is being invalidate because project 90e42435d75742019aaff626dac68768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:12.223370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.224408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a02e6359-a9b2-49a4-9482-b480f30e2e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:12.225893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1067/2119] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:11 2022] DELETE /identity/v3/projects/90e42435d75742019aaff626dac68768 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:12.228605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1053/2120] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:12.231319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631510-bc23-4ab8-b037-c0faed558628 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.231606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631510-bc23-4ab8-b037-c0faed558628 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.231882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631510-bc23-4ab8-b037-c0faed558628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.236883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.237271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.237550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.254673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9b631510-bc23-4ab8-b037-c0faed558628 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.256168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9b631510-bc23-4ab8-b037-c0faed558628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:12.258615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.259832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:12.272516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1068/2121] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:12.274519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.274938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.275283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.275783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1054/2122] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:12.279889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.299120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd714-9dde-41df-9318-e27d77a0cdbe None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.299237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.299362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.299481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:12.299572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.299572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.299572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.299911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:12.300323 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.301930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.306677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1069/2123] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:12.311746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.318392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1055/2124] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:12.321267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d95806-c9e0-499a-88f0-70f9bd10055e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.322871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.327816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.327816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.327816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.327816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.328197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:12.328197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.329753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.332683 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efb1ee-e756-4f12-b0d4-08128df4ca58 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.337754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.338346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.338630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.338925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.339547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.341511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.342018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1070/2125] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:12.346539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.357597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e78e-3b03-4ccd-8b71-6c9d8b2149e8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.362389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.363029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.363309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.363578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.364203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.366111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.440259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1056/2126] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:12.444469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.469073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1071/2127] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:12.476041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.517333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265769d1-d432-4c44-b9cd-c5907f6447e6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.550392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07011-2e4f-430a-8fd4-06d469dc1af3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.555455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.556175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.556459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.556776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.557423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.559530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.592716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.593333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.593609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.593880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.594505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.596521 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.669408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1072/2128] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:12.673894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.683934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b3d3-0234-491d-a996-f3be6dca4771 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.688894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.689597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.689881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.690156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.690994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.692629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.702691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1073/2129] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:12.706595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.709262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1057/2130] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:12.716535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.716933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5310e0-1981-43d4-b00b-22e7d90150aa admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.721967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.722507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.722823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.723100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.723745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.725454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.726672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bf140-5b8f-4a92-b1de-64ed21b1e5b6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.732080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.732680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.732963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.733237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.733891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.737847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.738057 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1074/2131] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:12.741317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.749535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1058/2132] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:12.750459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.755288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470eebb-1c1e-4893-a3d2-d73e17458bd3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.760637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf8545-e869-493e-90de-1d15f05e66ff admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.761385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.762265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.762556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.763118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] PATH_INFO: `/v3/projects/17c864ddf39840d19bd7565b5adddb05/users/59d6279b644846f2a5d226937988443f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.766072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.767588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.770530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.770530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.770530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.770530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.782242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c864ddf39840d19bd7565b5adddb05, user_id=59d6279b644846f2a5d226937988443f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.782897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1059/2133] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:12.785578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05217a46-7edf-4bb0-9763-5e27d12ef175 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.787601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:12.799143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e116-9060-4293-80bf-0611713e8f0e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:12.804090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:12.804796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.805073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.805349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] PATH_INFO: `/v3/projects/86da94ac86294ef492d994b53c814167/users/f71513118e7e4306af86121e38328483/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.824233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86da94ac86294ef492d994b53c814167, user_id=f71513118e7e4306af86121e38328483, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:12.827106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:12.829992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b67ef0-d642-4193-a3e9-13fd0787d273 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:12.879072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1075/2134] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:12 2022] PUT /identity/v3/projects/17c864ddf39840d19bd7565b5adddb05/users/59d6279b644846f2a5d226937988443f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:12.889894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.890200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.890573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.911071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1060/2135] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:12 2022] PUT /identity/v3/projects/86da94ac86294ef492d994b53c814167/users/f71513118e7e4306af86121e38328483/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:12.922704 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:12.923001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:12.923353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:12.937701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] MFA Rules not processed for user `59d6279b644846f2a5d226937988443f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.940686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:12.955262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] MFA Rules not processed for user `f71513118e7e4306af86121e38328483`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:12.956624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:13.007241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1076/2136] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:13.012239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.030376 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1061/2137] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:13.034928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.084252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e44804-a016-41a5-8406-bd10d137cef0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.090799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.091449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.092213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.092213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.092609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.094544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.104307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3047-720e-4f5a-9ae4-d260e5becfd9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.111342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.111963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.112242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.112517 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.113147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.115154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.137524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:13.167576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1077/2138] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:13.171584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.193215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1062/2139] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:13.194688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.253269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc1fa-81d4-42d8-8bdd-dff248775dd9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.276506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c255-73c1-438d-bb41-bc2541aca039 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.282353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.283180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.283464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.283795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.284790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.289064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.334329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.335011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.335301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.335569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.336241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.338214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.411582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1063/2140] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:13.415978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.426108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e51840-3075-4360-b423-88e918508c27 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.432110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.444015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.444015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.444015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.444015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.444015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.444512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1078/2141] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:13.446600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.446756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1064/2142] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:13.451973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.461716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b71e48-4099-4e69-b2ab-d1c77f7eb865 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.461894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d7a6-7dae-48c7-a724-423a4c7912ef admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.466298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.466891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.467012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.467239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.467510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] PATH_INFO: `/v3/projects/9a1947302f7a43a992eb78e9b6557927/users/cfa4f2dfed0a461c904df0a477d6002d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.467610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.467796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.468072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.468720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.470344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.483985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1079/2143] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:13.485453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1947302f7a43a992eb78e9b6557927, user_id=cfa4f2dfed0a461c904df0a477d6002d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.485599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.488285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.495744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2d0c-f72d-45dd-a9b8-979b023f905f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.500956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.501671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.501951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.502226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] PATH_INFO: `/v3/projects/c42d5213ec4143aeb75accac18c5b6f0/users/81a2c91514db4935b7e813295a00ef2e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.518595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42d5213ec4143aeb75accac18c5b6f0, user_id=81a2c91514db4935b7e813295a00ef2e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.521320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.525242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:13.634805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1065/2144] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:13 2022] PUT /identity/v3/projects/9a1947302f7a43a992eb78e9b6557927/users/cfa4f2dfed0a461c904df0a477d6002d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:13.639497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.656541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1080/2145] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:13 2022] PUT /identity/v3/projects/c42d5213ec4143aeb75accac18c5b6f0/users/81a2c91514db4935b7e813295a00ef2e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:13.660348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.713855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa728d-b104-4278-af46-df2f06041b20 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.735300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df07a7-2d1a-4f85-b678-cbebbf44679c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.740632 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.741359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.741706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.742039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.742892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.744781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.756287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1081/2146] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:13.760115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.771015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439c69-dc4b-409b-9625-5527a95b0e2c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.776567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.777227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.777505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.777781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.778011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.778441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.778913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.779306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.779676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] PATH_INFO: `/v3/projects/c42d5213ec4143aeb75accac18c5b6f0/users/81a2c91514db4935b7e813295a00ef2e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.780190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.782084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42d5213ec4143aeb75accac18c5b6f0, user_id=81a2c91514db4935b7e813295a00ef2e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.785382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2172f6-cd43-4c21-9fe3-b8a871640fec admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.793974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1066/2147] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:13.798311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.802082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1082/2148] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:13 2022] PUT /identity/v3/projects/c42d5213ec4143aeb75accac18c5b6f0/users/81a2c91514db4935b7e813295a00ef2e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:13.812437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.812720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.812999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.813509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52e57-b31c-4434-b0d3-da8521865929 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:13.841062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:13.853986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] MFA Rules not processed for user `81a2c91514db4935b7e813295a00ef2e`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:13.855318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:13.888822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:13.889524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.889810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.890089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] PATH_INFO: `/v3/projects/9a1947302f7a43a992eb78e9b6557927/users/cfa4f2dfed0a461c904df0a477d6002d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.892090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1947302f7a43a992eb78e9b6557927, user_id=cfa4f2dfed0a461c904df0a477d6002d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:13.894900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de8981-e91b-459a-806c-89aa8e307cfe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:13.906942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1067/2149] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:13 2022] PUT /identity/v3/projects/9a1947302f7a43a992eb78e9b6557927/users/cfa4f2dfed0a461c904df0a477d6002d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:13.918887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:13.919172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:13.919450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:13.925284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1083/2150] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:13.942397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:13.944181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:13.949598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] MFA Rules not processed for user `cfa4f2dfed0a461c904df0a477d6002d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:13.951388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:14.022943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17a12-ba5e-40dd-ae06-41c967688070 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:14.060443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1068/2151] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:14.073099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:14.076346 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:14.096435 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:14.097357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.097748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.098138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.142359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0050384-af9a-416a-a84b-e31a3a72f863 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:14.147407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:14.148074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.148363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.148743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.160299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:14.161482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:14.164730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:14.166477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:14.173871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1069/2152] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:14.179661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1084/2153] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:14.207373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:14.273275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cc64-eb86-4ead-923d-b6ca4de7ef75 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:14.277855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:14.278480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.278801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.279073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] PATH_INFO: `/v3/users/f71513118e7e4306af86121e38328483` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.280239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71513118e7e4306af86121e38328483)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:14.282154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:14.463213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:14.527541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:14.543559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71513118e7e4306af86121e38328483'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:14.545653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d4dc9-4315-434b-b2ae-07f1169f5d84 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:14.549511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:14.550320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.550766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.551049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.612432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:14.613931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b6b38-f8e1-4549-a2fb-f61a513e292e service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:14.623890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1085/2154] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:14.637551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71513118e7e4306af86121e38328483'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:14.647545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71513118e7e4306af86121e38328483'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:14.648566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71513118e7e4306af86121e38328483'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:14.666530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71513118e7e4306af86121e38328483'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:14.680484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1070/2155] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:14 2022] DELETE /identity/v3/users/f71513118e7e4306af86121e38328483 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:14.680719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.680904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.681350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.695225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:14.706711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:14.708202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:14.787388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:14.812579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1086/2156] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:14.824805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:14.826679 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:14.941371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d25a-c42b-4038-bc85-129df312ab67 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:14.951619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:14.954861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.954861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.954861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.964070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:14.966552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:14.978938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1071/2157] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:15.266759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:15.655386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:15.767736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:15.772951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:15.808648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca81c6-b7c8-46e4-a480-265fec2e7504 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:15.819274 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:15.820878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:15.821600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:15.822105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] PATH_INFO: `/v3/projects/86da94ac86294ef492d994b53c814167` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:15.825370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86da94ac86294ef492d994b53c814167)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:15.829193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:16.016301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86da94ac86294ef492d994b53c814167', 'request_id': 'req-e4e055f3-1953-4200-b0b3-b21b05b86329'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.024219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86da94ac86294ef492d994b53c814167', 'request_id': 'req-e4e055f3-1953-4200-b0b3-b21b05b86329'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.169145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86da94ac86294ef492d994b53c814167', 'request_id': 'req-e4e055f3-1953-4200-b0b3-b21b05b86329'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.189936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] The token cache is being invalidate because project 86da94ac86294ef492d994b53c814167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:16.190235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.190973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.192607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1087/2158] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:15 2022] DELETE /identity/v3/projects/86da94ac86294ef492d994b53c814167 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:16.198822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:16.212606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a17f-c4ea-4ed1-93f6-3d917f293a63 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:16.302735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:16.303986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:16.304572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:16.305083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] PATH_INFO: `/v3/users/cfa4f2dfed0a461c904df0a477d6002d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:16.307060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa4f2dfed0a461c904df0a477d6002d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:16.310822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:16.718474 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa4f2dfed0a461c904df0a477d6002d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.847437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa4f2dfed0a461c904df0a477d6002d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.866938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa4f2dfed0a461c904df0a477d6002d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.866938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa4f2dfed0a461c904df0a477d6002d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.893070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-33d84e5a-7bec-4466-9aac-3b0ca1394411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa4f2dfed0a461c904df0a477d6002d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:16.906063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1072/2159] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:16 2022] DELETE /identity/v3/users/cfa4f2dfed0a461c904df0a477d6002d => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:17.018081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:17.725439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:17.729557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:17.779193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e055f3-1953-4200-b0b3-b21b05b86329 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:17.920740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:17.921490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:17.921820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:17.922194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] PATH_INFO: `/v3/projects/9a1947302f7a43a992eb78e9b6557927` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:17.923364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1947302f7a43a992eb78e9b6557927)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:17.925634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:18.118679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:18.148023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1947302f7a43a992eb78e9b6557927', 'request_id': 'req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:18.178160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1947302f7a43a992eb78e9b6557927', 'request_id': 'req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:18.280572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1947302f7a43a992eb78e9b6557927', 'request_id': 'req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:18.299771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] The token cache is being invalidate because project 9a1947302f7a43a992eb78e9b6557927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:18.300099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:18.300846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-89b31bbd-aec0-4a41-bcd4-2dc731b4b0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:18.302383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1088/2160] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:17 2022] DELETE /identity/v3/projects/9a1947302f7a43a992eb78e9b6557927 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:18.307848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:18.308954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:18.309124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:18.345881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:18.347332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:18.429448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1073/2161] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:18.438040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:18.438328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:18.438604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:18.460705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:18.461900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:18.475034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1089/2162] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:18.477808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:18.496606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356c896-a8df-4dc5-a1e7-d035866ca0ea None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:18.505080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:18.505728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:18.506006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:18.506297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:18.507026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:18.507326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:18.509393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:18.546633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1074/2163] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:18.547857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:18.559023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae8d8-8047-4d24-967d-49d2202548fb None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:18.564397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:18.568197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:18.568197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:18.568197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:18.568197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:18.568197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:18.832587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1090/2164] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:18 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:18.838239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:18.855049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866767-4148-4a34-af9d-47e237d62fd7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:18.909073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:18.930607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:18.931290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:18.931567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:18.931843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:18.932482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:18.935220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.083904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1075/2165] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:18 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:19.095031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:19.112934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7ae8f-c22c-4f82-bdf0-54b9392abd04 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:19.118220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:19.119427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.119427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.120101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.125133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:19.133710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.154909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1091/2166] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:19.159071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:19.172598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91661e2a-6d4c-46c7-935f-bda91fce5397 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:19.177405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:19.178029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.180552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.180552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.180552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:19.181556 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce525dd-2abc-4d6c-9a9f-28f7ffd091a4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.194709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1076/2167] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:19.198719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:19.210311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1756727-8c4c-49b9-b1f2-91bf5391fab3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:19.215843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:19.216687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.217034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.217405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] PATH_INFO: `/v3/projects/6ceb3ec1a07e4e3aa8126a9b514b37a2/users/b5553ec3e93c4510969d5e76ce7bc345/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.240142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceb3ec1a07e4e3aa8126a9b514b37a2, user_id=b5553ec3e93c4510969d5e76ce7bc345, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:19.243028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.339186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1092/2168] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:19 2022] PUT /identity/v3/projects/6ceb3ec1a07e4e3aa8126a9b514b37a2/users/b5553ec3e93c4510969d5e76ce7bc345/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:19.352697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.353565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.353565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.396041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] MFA Rules not processed for user `b5553ec3e93c4510969d5e76ce7bc345`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:19.397793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:19.497005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1077/2169] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:19.498179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:19.598947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c7cfd-7b31-47e9-b536-083cd359b42a admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:19.603989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:19.604600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.604911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.605697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.606328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:19.608517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.771712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1093/2170] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:19 2022] POST /identity/v3/projects => generated 381 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:19.773457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:19.798697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b05ed-e1fb-4811-bb85-baaad8f49a04 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:19.869489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:19.870328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:19.870554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:19.870891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:19.871515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:19.873637 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:19.998737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1078/2171] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:19 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:20.002842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:20.014447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1452a-4c4a-42ea-b0be-63d621396a6e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:20.024037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:20.028680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:20.028680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:20.028680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:20.029442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:20.033267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:20.055394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1094/2172] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:20.060821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:20.073470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d830-fc56-4858-807a-58f712f4438f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:20.086253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:20.087192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:20.087823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:20.088397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] PATH_INFO: `/v3/projects/95d3ec7e7b544d28b405915112d3d395/users/dca46603653445b697f1805f51674a9e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:20.116164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d3ec7e7b544d28b405915112d3d395, user_id=dca46603653445b697f1805f51674a9e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:20.129432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:20.252238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1079/2173] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:20 2022] PUT /identity/v3/projects/95d3ec7e7b544d28b405915112d3d395/users/dca46603653445b697f1805f51674a9e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:20.257240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:20.275999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8895d-e760-4796-85ef-d995d3612ee2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:20.361149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:20.362079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:20.362582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:20.363847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:20.364596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:20.366854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964b0bb-2b99-4261-85c1-fc1396ceef3b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:20.382866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1095/2174] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:20.387959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:20.402843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8b3f-2abf-48c0-9394-141c28158dbe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:20.410530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:20.415815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:20.415815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:20.415815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] PATH_INFO: `/v3/projects/95d3ec7e7b544d28b405915112d3d395/users/dca46603653445b697f1805f51674a9e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:20.427911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d3ec7e7b544d28b405915112d3d395, user_id=dca46603653445b697f1805f51674a9e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:20.433894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:20.468538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1080/2175] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:20 2022] PUT /identity/v3/projects/95d3ec7e7b544d28b405915112d3d395/users/dca46603653445b697f1805f51674a9e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:20.481537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:20.482616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:20.483147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:20.529885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] MFA Rules not processed for user `dca46603653445b697f1805f51674a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:20.531745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:20.643890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1096/2176] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:21.171210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:21.174369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:21.244134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414319-37fe-49ea-b203-0c141ec608a4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:21.248770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:21.252446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:21.252446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:21.252446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:21.261805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:21.263387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:21.277369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1081/2177] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:21.966211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:21.971880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cac5d6-7337-46bb-8cbb-85da2483b268 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:21.977494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:21.978219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:21.978567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:21.978943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:22.048553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:22.051808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:22.063217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1097/2178] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:22.204432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:23.140684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:25.172830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.175830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:25.277653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab178a-f840-43dc-b482-94161585a423 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:25.300235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:25.301524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.301975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.302438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] PATH_INFO: `/v3/users/59d6279b644846f2a5d226937988443f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.304163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d6279b644846f2a5d226937988443f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:25.306487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:25.559924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d6279b644846f2a5d226937988443f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.676190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d6279b644846f2a5d226937988443f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.697506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d6279b644846f2a5d226937988443f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.698942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.699734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d6279b644846f2a5d226937988443f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.723684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657efd18-bca5-48d9-9144-831665ff9c4e service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:25.739301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:25.739896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0169948f-c2b5-47b0-947f-4ae32b1c6476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d6279b644846f2a5d226937988443f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.740346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.740815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.741280 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] PATH_INFO: `/v3/users/b5553ec3e93c4510969d5e76ce7bc345` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.743484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5553ec3e93c4510969d5e76ce7bc345)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:25.747791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:25.762347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1082/2179] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:25 2022] DELETE /identity/v3/users/59d6279b644846f2a5d226937988443f => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:25.770365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.770773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.770994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.794374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:25.795689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:25.895988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1083/2180] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:25.908721 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.910436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.011265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bc962-641c-4ec4-be3e-4bdfb9f9951e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.018087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.019190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.019573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.020076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.025958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5553ec3e93c4510969d5e76ce7bc345'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.027394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5553ec3e93c4510969d5e76ce7bc345'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.037907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.040961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af1fb-8c56-4695-bc0d-34f452724cc1 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.059205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1084/2181] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:26.109162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5553ec3e93c4510969d5e76ce7bc345'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.111993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.125070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5553ec3e93c4510969d5e76ce7bc345'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.141638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5553ec3e93c4510969d5e76ce7bc345'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.149181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1098/2182] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:25 2022] DELETE /identity/v3/users/b5553ec3e93c4510969d5e76ce7bc345 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:26.154271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.154551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.154865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.180883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:26.182134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.269345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1085/2183] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:26.279193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.281269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.298055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.412968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83739f-93e7-4a7c-829b-99bde5c3ebd9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.417261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.417901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.418183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.418454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.433934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.437721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.461995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1099/2184] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:26.652978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.654804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.671829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c47c2-bee0-406e-ad61-272e9ad6d816 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.688614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.690928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.693289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.693289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] PATH_INFO: `/v3/projects/17c864ddf39840d19bd7565b5adddb05` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.697187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c864ddf39840d19bd7565b5adddb05)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.709772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.934043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c864ddf39840d19bd7565b5adddb05', 'request_id': 'req-028803ad-fd3f-4eec-96e3-59c6f474d140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:26.941159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c864ddf39840d19bd7565b5adddb05', 'request_id': 'req-028803ad-fd3f-4eec-96e3-59c6f474d140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.011183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c864ddf39840d19bd7565b5adddb05', 'request_id': 'req-028803ad-fd3f-4eec-96e3-59c6f474d140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.046913 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:27.049285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] The token cache is being invalidate because project 17c864ddf39840d19bd7565b5adddb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:27.050781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.051635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.056676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1086/2185] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:26 2022] DELETE /identity/v3/projects/17c864ddf39840d19bd7565b5adddb05 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:27.059141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.088802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2262b-8936-4bc9-b2a1-c7a12f13de52 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.208379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.209041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.209330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.209629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] PATH_INFO: `/v3/users/81a2c91514db4935b7e813295a00ef2e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.210850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a2c91514db4935b7e813295a00ef2e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.212856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:27.471014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.472760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:27.487751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028803ad-fd3f-4eec-96e3-59c6f474d140 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.495890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.496669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.497024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.497368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] PATH_INFO: `/v3/projects/6ceb3ec1a07e4e3aa8126a9b514b37a2` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.502117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ceb3ec1a07e4e3aa8126a9b514b37a2)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.508049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:27.555454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a2c91514db4935b7e813295a00ef2e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.559564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a2c91514db4935b7e813295a00ef2e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.660550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a2c91514db4935b7e813295a00ef2e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.692545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a2c91514db4935b7e813295a00ef2e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.717732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ceb3ec1a07e4e3aa8126a9b514b37a2', 'request_id': 'req-47ae2096-f059-45f6-a29d-324595af5157'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.730442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ceb3ec1a07e4e3aa8126a9b514b37a2', 'request_id': 'req-47ae2096-f059-45f6-a29d-324595af5157'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.730825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:27.735046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a2c91514db4935b7e813295a00ef2e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.744224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1100/2186] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:27 2022] DELETE /identity/v3/users/81a2c91514db4935b7e813295a00ef2e => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:27.899553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ceb3ec1a07e4e3aa8126a9b514b37a2', 'request_id': 'req-47ae2096-f059-45f6-a29d-324595af5157'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.940807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] The token cache is being invalidate because project 6ceb3ec1a07e4e3aa8126a9b514b37a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:27.941519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.945251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.955986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1087/2187] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:27 2022] DELETE /identity/v3/projects/6ceb3ec1a07e4e3aa8126a9b514b37a2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:27.960136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.962503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:28.097768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d79cc-7d8e-413a-927a-ca34440bd14c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:28.105065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:28.106949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:28.107466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:28.107931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] PATH_INFO: `/v3/users/dca46603653445b697f1805f51674a9e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:28.109607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca46603653445b697f1805f51674a9e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:28.112170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:28.415294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca46603653445b697f1805f51674a9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:28.415961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca46603653445b697f1805f51674a9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:28.507541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca46603653445b697f1805f51674a9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:28.516028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca46603653445b697f1805f51674a9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:28.529429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca46603653445b697f1805f51674a9e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:28.535990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1101/2188] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:27 2022] DELETE /identity/v3/users/dca46603653445b697f1805f51674a9e => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:28.584590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:28.586103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:28.710079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2096-f059-45f6-a29d-324595af5157 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:28.795333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:28.795968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:28.796248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:28.796522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] PATH_INFO: `/v3/projects/c42d5213ec4143aeb75accac18c5b6f0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:28.797650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42d5213ec4143aeb75accac18c5b6f0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:28.799596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:28.990540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42d5213ec4143aeb75accac18c5b6f0', 'request_id': 'req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.007310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42d5213ec4143aeb75accac18c5b6f0', 'request_id': 'req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.589550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42d5213ec4143aeb75accac18c5b6f0', 'request_id': 'req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.604769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:29.608301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:29.624483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e7a0-a553-4ed1-b871-56870f270189 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:29.635176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] The token cache is being invalidate because project c42d5213ec4143aeb75accac18c5b6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:29.636445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.637023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e1ceb53a-24a3-42b0-8ce7-256b1d386fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.639787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1088/2189] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:28 2022] DELETE /identity/v3/projects/c42d5213ec4143aeb75accac18c5b6f0 => generated 0 bytes in 1056 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:29.645151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1957e-c03d-4f5a-8212-e5d74f074ca4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.645791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1957e-c03d-4f5a-8212-e5d74f074ca4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.647933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1957e-c03d-4f5a-8212-e5d74f074ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.682751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4ce1957e-c03d-4f5a-8212-e5d74f074ca4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:29.684038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4ce1957e-c03d-4f5a-8212-e5d74f074ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:29.718338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:29.719060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.719341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.719615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] PATH_INFO: `/v3/projects/95d3ec7e7b544d28b405915112d3d395` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.720910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d3ec7e7b544d28b405915112d3d395)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:29.722990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:29.769448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1089/2190] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:29.778189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.778479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.778796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.801256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:29.803735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:29.812919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1090/2191] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:29.817494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:29.830842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0727c3-ef50-45e6-8347-8af9c95a5817 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:29.838810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:29.840025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.840402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.840886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.842466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:29.843077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:29.847405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:29.874748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1091/2192] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:29.879234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:29.901393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94988e-ab2a-48fc-b8aa-1d92a619ce77 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:29.908374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:29.909312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.909841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.910956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.917585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:29.919780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f9b08-fc0f-4557-8c10-1316f3f694bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:29.929748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d3ec7e7b544d28b405915112d3d395', 'request_id': 'req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.939314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d3ec7e7b544d28b405915112d3d395', 'request_id': 'req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.131033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d3ec7e7b544d28b405915112d3d395', 'request_id': 'req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.150130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1092/2193] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:29 2022] POST /identity/v3/projects => generated 383 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:30.151995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf75c-9f37-4a0c-8168-06ddcce389c3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.152639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf75c-9f37-4a0c-8168-06ddcce389c3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.153275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf75c-9f37-4a0c-8168-06ddcce389c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.158589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] The token cache is being invalidate because project 95d3ec7e7b544d28b405915112d3d395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:30.159056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.168942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.171968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1102/2194] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:29 2022] DELETE /identity/v3/projects/95d3ec7e7b544d28b405915112d3d395 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:30.176063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.176063 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.176446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f9caf75c-9f37-4a0c-8168-06ddcce389c3 None None] MFA Rules not processed for user `f7f0342a701f492a9de96588c1569fac`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:30.177241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f9caf75c-9f37-4a0c-8168-06ddcce389c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.243974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1093/2195] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:30.246116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521576-8903-4f64-a0b0-3dc16cb64b26 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.246411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521576-8903-4f64-a0b0-3dc16cb64b26 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.246732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521576-8903-4f64-a0b0-3dc16cb64b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.252094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16c8e-30cd-4ef0-abab-5f6faba69c11 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.267708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:30.269735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4e521576-8903-4f64-a0b0-3dc16cb64b26 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:30.271295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4e521576-8903-4f64-a0b0-3dc16cb64b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.344511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.345647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.347993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.347993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.347993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.350146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.368759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1094/2196] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:30.377223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.377494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.377655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.404420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:30.407708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.434708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1095/2197] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:30.438817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.457587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d5d93-e2da-4175-b1a3-7380560ba3f6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.465812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.466987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.467457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.467940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.468974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:30.469475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.472716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.484053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1103/2198] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:30.488726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.514821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d46e6-d1cf-4575-acce-f7d85eb2beaf admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.517832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1096/2199] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:30.524422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.530371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.531301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.532270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.532417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.533396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.535761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.548877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611274a-6836-4264-be07-abb0ac833a4e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.569635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1104/2200] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:30.574559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.577446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.578947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.579258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.581865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.582864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.584853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.598328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec461f6-3059-4497-adcd-43e376873cd8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.605453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.606585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.607054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.607623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.608801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.612296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.658229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1105/2201] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:30.658229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.658229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62a45c-17b3-4201-8c79-f8aa9151bed6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.662712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.663527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.664002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.665375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] PATH_INFO: `/v3/projects/5f76adbf7fb2442e8f1e86beaf4a0b5b/users/f34c0ec5c3fa4e2e90a2754121c82037/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.683995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f76adbf7fb2442e8f1e86beaf4a0b5b, user_id=f34c0ec5c3fa4e2e90a2754121c82037, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.687011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7fed3-42a9-49ab-933b-91a82022624a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.837803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1097/2202] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/projects => generated 383 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:30.841344 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.871406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1106/2203] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:30 2022] PUT /identity/v3/projects/5f76adbf7fb2442e8f1e86beaf4a0b5b/users/f34c0ec5c3fa4e2e90a2754121c82037/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:30.884193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.884525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.884843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.937146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] MFA Rules not processed for user `f34c0ec5c3fa4e2e90a2754121c82037`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:30.938787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.972676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928844-612d-469d-b959-ab174b153f87 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.078359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1107/2204] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:31.097585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:31.099960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:31.111251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.111887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.112174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.112446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.113098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.115223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.197773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4f72-2686-4d14-9a07-218b6f6acd26 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.205398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.206399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.206950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.207442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.224947 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.227597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.235015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1098/2205] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:30 2022] POST /identity/v3/users => generated 292 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:31.239401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:31.247045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1108/2206] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:31.249694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:31.265596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4645e0-e7a0-497d-965d-bfb36ff7476e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.292941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.296005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.297331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.298639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.301680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.311722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.342009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1099/2207] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:31.348216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:31.372921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0967e-1951-4cb2-9ee7-c55fcee7ab77 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.413336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.423200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.424127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.424721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.428608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.436863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f3ab5-4459-4362-a6a6-4f8974dd2ccc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.483626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1109/2208] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:31.487784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:31.507595 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a123f5-931a-44b9-9c59-d5461a2fde15 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.525847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.525847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.525847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.525847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] PATH_INFO: `/v3/projects/e6e3ea04872b426ebce8e854a4b574b1/users/1b3c50092e22477ca1b736104ae6bcf7/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.586164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e3ea04872b426ebce8e854a4b574b1, user_id=1b3c50092e22477ca1b736104ae6bcf7, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.596087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.718769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1100/2209] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:31 2022] PUT /identity/v3/projects/e6e3ea04872b426ebce8e854a4b574b1/users/1b3c50092e22477ca1b736104ae6bcf7/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:31.732511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.733470 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.734000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.864019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] MFA Rules not processed for user `1b3c50092e22477ca1b736104ae6bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:31.867369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:32.098199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1110/2210] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:32.103473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:32.200590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b150-51b3-4de3-8c0f-cb8095c75e4a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.207299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.208050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.208412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.208800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.209610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.212235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:32.426564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1101/2211] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:32 2022] POST /identity/v3/projects => generated 381 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:32.429003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:32.456567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c89aa1-c7a3-4348-8c6a-2c2fad0edd81 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.545965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.546860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.547261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.547725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.548603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.557879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:32.750572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1111/2212] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:32 2022] POST /identity/v3/users => generated 297 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:32.752075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:32.774612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ae77c-bb65-4f20-8761-1ccbc03bc405 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.785098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.786319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.787259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.787995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.789253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.793044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:32.815911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1102/2213] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:32.823952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:32.848365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c220-b3b0-45ea-8512-df1f503d7ac7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.857150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.858248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.859049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.860003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] PATH_INFO: `/v3/projects/42d94da85d3b4218863df2172c887662/users/0a9e29353f9042ea8f4ce89ba13ff09e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.872439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:32.897031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d94da85d3b4218863df2172c887662, user_id=0a9e29353f9042ea8f4ce89ba13ff09e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.901186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.014433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1112/2214] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:32 2022] PUT /identity/v3/projects/42d94da85d3b4218863df2172c887662/users/0a9e29353f9042ea8f4ce89ba13ff09e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:33.021132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.045045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d638-1266-4b8e-9889-c49bcff6989c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.173799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.185097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.185097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.185097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.185097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.185097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f9d72-acf3-47a5-99ad-cb693bf0dfa9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.208707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1103/2215] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:33.217020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.235288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f56ea-7222-4f92-9c12-178826f55a95 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.249105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.250529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.251106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.251686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] PATH_INFO: `/v3/projects/42d94da85d3b4218863df2172c887662/users/0a9e29353f9042ea8f4ce89ba13ff09e/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.256088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d94da85d3b4218863df2172c887662, user_id=0a9e29353f9042ea8f4ce89ba13ff09e, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.262974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.286863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1113/2216] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:33 2022] PUT /identity/v3/projects/42d94da85d3b4218863df2172c887662/users/0a9e29353f9042ea8f4ce89ba13ff09e/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:33.299143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.300769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.301391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.308561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:33.340671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] MFA Rules not processed for user `0a9e29353f9042ea8f4ce89ba13ff09e`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:33.342148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:33.499139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1104/2217] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:33.514014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.515975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:33.620711 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4746dc-2d53-4c56-b2f8-2b7142136cec admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.624655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.625289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.625601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.625882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.639828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.642188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.659592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1114/2218] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:33.911414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.914203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:33.924503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.948795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ab5cb-e62a-4022-b574-3aa8122f0677 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.958317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.961078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.962695 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.964142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.020096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ffac8-2541-4bc2-890d-76e406dc375a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.028163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.030554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.030970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.031402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.034770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:34.067806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.070107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ec3aa-66e0-4e0b-8214-4a7daac8e377 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.087324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1115/2219] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:34.154318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.155886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.174926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1105/2220] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:34.670629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bcb8-8209-4447-a852-14a2625d49bc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.672545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bcb8-8209-4447-a852-14a2625d49bc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.673273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bcb8-8209-4447-a852-14a2625d49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.706348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4f77bcb8-8209-4447-a852-14a2625d49bc None None] MFA Rules not processed for user `3f9025a5d28b47d8bc36f131e908382f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:34.707630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4f77bcb8-8209-4447-a852-14a2625d49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:34.802113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1116/2221] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:34.996002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.085411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992443fd-c93d-44cc-b979-5aa1a8d30644 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.090216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.090927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.091544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.091859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] PATH_INFO: `/v3/users/1b3c50092e22477ca1b736104ae6bcf7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.093241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3c50092e22477ca1b736104ae6bcf7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.095668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.151514 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:35.371337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3c50092e22477ca1b736104ae6bcf7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:35.450802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3c50092e22477ca1b736104ae6bcf7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:35.461075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3c50092e22477ca1b736104ae6bcf7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:35.462022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3c50092e22477ca1b736104ae6bcf7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:35.484351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3c50092e22477ca1b736104ae6bcf7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:35.501420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1106/2222] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:34 2022] DELETE /identity/v3/users/1b3c50092e22477ca1b736104ae6bcf7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:35.503297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.510797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.510797 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.542920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:35.543927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:35.594046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:35.609324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.610988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:35.653011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1117/2223] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:35.673066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.675549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:35.759434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22cc1e-63c4-48e1-bb9c-2ed90c5fdb75 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.767329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.769221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.769493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.769770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] PATH_INFO: `/v3/users/f34c0ec5c3fa4e2e90a2754121c82037` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.770999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f34c0ec5c3fa4e2e90a2754121c82037)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.773378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.796950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d7561-ac96-45d9-b3c5-9109b3a966d1 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.802070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.804105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.805172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.805552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.817272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.819671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363bc15-3b40-40f5-8cc1-bb2806bc6e06 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.831682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1118/2224] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:36.068306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f34c0ec5c3fa4e2e90a2754121c82037'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.174206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f34c0ec5c3fa4e2e90a2754121c82037'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.184371 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f34c0ec5c3fa4e2e90a2754121c82037'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.185158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f34c0ec5c3fa4e2e90a2754121c82037'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.218477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f34c0ec5c3fa4e2e90a2754121c82037'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.232621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1107/2225] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:35 2022] DELETE /identity/v3/users/f34c0ec5c3fa4e2e90a2754121c82037 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:36.239108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:36.239560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:36.239945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:36.288599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:36.290145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:36.485627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1119/2226] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:36.504405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:36.506987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:36.674946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:36.677419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:36.696677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70166a58-ca43-4c99-bec6-1194a4b630d8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:36.701546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:36.702639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:36.703593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:36.704286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:36.716633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:36.717305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa9b29-8198-44bc-a534-f5801223a2ed None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:36.720639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:36.732598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:36.733387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:36.735003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:36.735297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] PATH_INFO: `/v3/projects/e6e3ea04872b426ebce8e854a4b574b1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:36.736431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1108/2227] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:36.738144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e3ea04872b426ebce8e854a4b574b1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:36.741116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:36.911367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e3ea04872b426ebce8e854a4b574b1', 'request_id': 'req-1d729d50-4e94-4ebb-909f-b2759e108890'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.921180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e3ea04872b426ebce8e854a4b574b1', 'request_id': 'req-1d729d50-4e94-4ebb-909f-b2759e108890'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:36.981144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:37.008483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e3ea04872b426ebce8e854a4b574b1', 'request_id': 'req-1d729d50-4e94-4ebb-909f-b2759e108890'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.029116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] The token cache is being invalidate because project e6e3ea04872b426ebce8e854a4b574b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:37.029542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.039398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.045605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1120/2228] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:36 2022] DELETE /identity/v3/projects/e6e3ea04872b426ebce8e854a4b574b1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:37.049784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:37.059931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:37.064200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d0ac0-968d-4f92-a765-e99228706424 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:37.180305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:37.180794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:37.181104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:37.181377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] PATH_INFO: `/v3/users/0a9e29353f9042ea8f4ce89ba13ff09e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:37.183257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9e29353f9042ea8f4ce89ba13ff09e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:37.186005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:37.508853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9e29353f9042ea8f4ce89ba13ff09e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.591220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9e29353f9042ea8f4ce89ba13ff09e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.600845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9e29353f9042ea8f4ce89ba13ff09e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.601905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9e29353f9042ea8f4ce89ba13ff09e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.615028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:37.617604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:37.632139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1bd6ebf-a999-4d33-a4b7-32cca5865ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9e29353f9042ea8f4ce89ba13ff09e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.670097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1109/2229] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:37 2022] DELETE /identity/v3/users/0a9e29353f9042ea8f4ce89ba13ff09e => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:37.670524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729d50-4e94-4ebb-909f-b2759e108890 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:37.730444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:37.731224 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:37.731577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:37.731926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] PATH_INFO: `/v3/projects/5f76adbf7fb2442e8f1e86beaf4a0b5b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:37.733172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f76adbf7fb2442e8f1e86beaf4a0b5b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:37.735367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:37.977898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f76adbf7fb2442e8f1e86beaf4a0b5b', 'request_id': 'req-a40933e4-efa9-4545-9782-e04d5770a6d6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:37.999305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f76adbf7fb2442e8f1e86beaf4a0b5b', 'request_id': 'req-a40933e4-efa9-4545-9782-e04d5770a6d6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.107576 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f76adbf7fb2442e8f1e86beaf4a0b5b', 'request_id': 'req-a40933e4-efa9-4545-9782-e04d5770a6d6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.128334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] The token cache is being invalidate because project 5f76adbf7fb2442e8f1e86beaf4a0b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:38.128634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.131700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a40933e4-efa9-4545-9782-e04d5770a6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.135796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1121/2230] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:37 2022] DELETE /identity/v3/projects/5f76adbf7fb2442e8f1e86beaf4a0b5b => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:38.140735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.141021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.141307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.168195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:38.173181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:38.283187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1110/2231] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:38.290821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.291664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.292382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.324904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:38.326461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:38.358855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1122/2232] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:38.364033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.375844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21433d-f5d7-4d04-9a0c-85ba6b324214 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:38.391863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:38.392516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.392883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.393193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.393896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:38.394234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:38.396620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:38.415989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.417944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:38.425770 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1111/2233] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:38.431705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.439162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c031e0-63bc-41c6-b238-254912aed5dc None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:38.456078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:38.457319 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.457939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.458423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] PATH_INFO: `/v3/projects/42d94da85d3b4218863df2172c887662` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.462076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d94da85d3b4218863df2172c887662)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:38.462264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d226371-8b88-4281-a824-bc67ec4542fb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:38.464157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:38.468985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:38.469750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.470003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.470229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.475106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:38.479522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:38.648661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d94da85d3b4218863df2172c887662', 'request_id': 'req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.658090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d94da85d3b4218863df2172c887662', 'request_id': 'req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.689920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1112/2234] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/projects => generated 383 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:38.692359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.748664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:38.805409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b29c-f008-4d02-99a3-b621944b7a03 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:38.810755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:38.811382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.811663 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.812002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.813258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:38.814634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:38.820716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d94da85d3b4218863df2172c887662', 'request_id': 'req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.835611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] The token cache is being invalidate because project 42d94da85d3b4218863df2172c887662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:38.836099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.837253 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-64c0fcbb-88f0-4bf4-b590-ea22f4d72be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:38.839691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1123/2235] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:38 2022] DELETE /identity/v3/projects/42d94da85d3b4218863df2172c887662 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:38.846174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d948-8375-40c5-aa99-2d0023b0f0b5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.846831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d948-8375-40c5-aa99-2d0023b0f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.847273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592d948-8375-40c5-aa99-2d0023b0f0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.873416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7592d948-8375-40c5-aa99-2d0023b0f0b5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:38.875254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7592d948-8375-40c5-aa99-2d0023b0f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:38.879650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:38.889031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1124/2236] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:38.893597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.893887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.894159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.915005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:38.916251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:38.924968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1125/2237] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:38.929419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.946816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72753ffc-3008-49b0-a8c3-d0136a968704 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:38.948629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:38.949437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:38.949733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:38.950015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:38.951336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:38.951336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:38.952934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:38.991961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1113/2238] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:38 2022] POST /identity/v3/users => generated 292 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:38.996418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:38.999516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:39.001172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1126/2239] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:39.005839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.024204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f969-586f-4db1-9473-a301b25cd73d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.026822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db20e-8e95-4eb2-9f07-4f9bcc41439c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.033466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.034392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.036768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.037240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.037681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.038158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.038441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.038729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.040072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.040669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.043374 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.043649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.059272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1114/2240] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:39.063430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.079687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef5d3c-d5c8-448c-8c39-8e63425f0700 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.087812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.089341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.089718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.090026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.090978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.093071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.122692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1115/2241] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:39.128359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.145173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee001ce9-4e29-4a2b-b0fe-f09a124b1bfc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.152158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.153834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.154286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.154746 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] PATH_INFO: `/v3/projects/e3fc853baefe4aa6b105874207810d01/users/a62220151af547aa9dbeb84f8e00c045/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.188768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1127/2242] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:39 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:39.189761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.203887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fc853baefe4aa6b105874207810d01, user_id=a62220151af547aa9dbeb84f8e00c045, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.214388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa68c6-ded7-44aa-9767-dfef4d86aef0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.324842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1116/2243] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:39 2022] PUT /identity/v3/projects/e3fc853baefe4aa6b105874207810d01/users/a62220151af547aa9dbeb84f8e00c045/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:39.327208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.328698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.331610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.333616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae8c9-7f42-453c-8384-30c7aa13ba18 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.390433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:39.391914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:39.489736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.490700 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.490999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.491279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.491939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.497981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.624140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1117/2244] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:39.642387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.644369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:39.753000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282a09d-1f68-4199-9633-199d92089a0f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.766490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1128/2245] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:39 2022] POST /identity/v3/users => generated 293 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:39.770566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.771596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.778499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.778863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.779216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.795360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874932-01d2-4511-aa0b-3e10fce785ec admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.808287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.809176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.812152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.813287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.814717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.815521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.815702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.819736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.852499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1118/2246] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:39.852838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1129/2247] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:39.858317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.873907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225f48-d324-473c-bf38-a3a4c8f4ca60 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.882622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.883849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.884215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.884575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.891772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.891772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e5dc8-0b9c-40f0-bd8b-5a8478c85faf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:39.908263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1119/2248] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:39.913061 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.929251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862bbf7-4177-470e-9ff7-52867043b8ae admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:39.936322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:39.937405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.937779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.938308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] PATH_INFO: `/v3/projects/032e20ffecdc448eb56efdf595c67738/users/45fc48d473ac4cc8b7e3932651420fdb/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.959563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:39.968263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e20ffecdc448eb56efdf595c67738, user_id=45fc48d473ac4cc8b7e3932651420fdb, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:39.972606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:40.058137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1130/2249] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:39 2022] PUT /identity/v3/projects/032e20ffecdc448eb56efdf595c67738/users/45fc48d473ac4cc8b7e3932651420fdb/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:40.078745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee5542-2ef6-4296-be9c-bef1d78755d5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:40.079448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee5542-2ef6-4296-be9c-bef1d78755d5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:40.079788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee5542-2ef6-4296-be9c-bef1d78755d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:40.128153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e6ee5542-2ef6-4296-be9c-bef1d78755d5 None None] MFA Rules not processed for user `45fc48d473ac4cc8b7e3932651420fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:40.129473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e6ee5542-2ef6-4296-be9c-bef1d78755d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:40.241152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1120/2250] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:40.264245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:40.266142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:40.361453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485951-1005-49b1-94d9-bf1edd029c86 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:40.361453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:40.361453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:40.361453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:40.362228 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:40.375064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:40.376757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1cc33-6c8c-4376-87ea-e230da6dcc35 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:40.390142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1131/2251] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:41.194378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:41.283305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:41.284146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:41.284640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:41.310059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] MFA Rules not processed for user `5171bf0514c3474aae358a9d7cfaf538`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:41.311627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:41.426491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1121/2252] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:42.112538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:42.313773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0080b3-5a1d-462e-956c-95f9f28e376c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:42.314233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0080b3-5a1d-462e-956c-95f9f28e376c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:42.314794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0080b3-5a1d-462e-956c-95f9f28e376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:42.362613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3d0080b3-5a1d-462e-956c-95f9f28e376c None None] MFA Rules not processed for user `a62220151af547aa9dbeb84f8e00c045`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:42.367397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3d0080b3-5a1d-462e-956c-95f9f28e376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:42.474625 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1132/2253] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:42.491206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:42.494951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:42.527805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790c4b0-b33b-4bfd-8eef-c85102429983 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:42.541787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:42.548431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:42.549033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:42.549460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:42.589386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:42.591347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9be6f-526f-4f83-af37-1393562a1373 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:42.603246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1122/2254] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:42.943076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:44.870727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:44.871535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:44.871535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:44.891432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] MFA Rules not processed for user `a62220151af547aa9dbeb84f8e00c045`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:44.892671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:44.901113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1133/2255] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:44.989505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:44.990819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:44.991046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:45.034789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:45.035880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:45.180821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1123/2256] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:45.248244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:45.252239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:45.379266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897be9c1-8ad1-41c1-a17d-f89895782e2c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:45.387595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:45.387937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:45.388287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:45.388635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:45.401508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:45.411384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc08f9-c593-4b5f-bf79-d91ff44aa9cd service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:45.430196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1134/2257] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:58:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:58:46.050676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:46.693988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:48.627141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:51.160106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.113569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.115199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.264053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64c08-1ace-4bb6-9914-6ff9fa8484ee None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.287919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.287919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.287919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.287919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] PATH_INFO: `/v3/users/6de08e32cefe42ddb5c05bf02778eeea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.288438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de08e32cefe42ddb5c05bf02778eeea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.288438 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.499119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de08e32cefe42ddb5c05bf02778eeea'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.623064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.659765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de08e32cefe42ddb5c05bf02778eeea'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.670715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de08e32cefe42ddb5c05bf02778eeea'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.671814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de08e32cefe42ddb5c05bf02778eeea'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.702717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de08e32cefe42ddb5c05bf02778eeea'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.714726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1124/2258] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:52 2022] DELETE /identity/v3/users/6de08e32cefe42ddb5c05bf02778eeea => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:52.719659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.720193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.720193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.765454 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:52.770895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.981738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1135/2259] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:53.052568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.054178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:53.254919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d124a5-c42c-4955-b77e-fbced560be34 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.285597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.287048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.287498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.287935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.310784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.312290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.341072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1125/2260] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:54.105284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:54.107276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:54.135446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614aac6c-da6f-49ad-a65c-24147242979a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:54.158364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:54.159782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:54.160440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:54.160969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] PATH_INFO: `/v3/users/45fc48d473ac4cc8b7e3932651420fdb` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:54.163301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45fc48d473ac4cc8b7e3932651420fdb)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:54.166849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:54.633454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:54.636182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:54.661962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70521c-74a5-45e5-9eec-00a6208d29a9 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:54.684026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:54.684910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:54.694905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:54.694905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] PATH_INFO: `/v3/projects/db4ed6100ce34e16bf75d9e8f5e5092d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:54.694905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4ed6100ce34e16bf75d9e8f5e5092d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:54.694905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:54.814291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45fc48d473ac4cc8b7e3932651420fdb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.815914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45fc48d473ac4cc8b7e3932651420fdb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.865009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4ed6100ce34e16bf75d9e8f5e5092d', 'request_id': 'req-de51f086-b987-47cb-ab3c-59b5a8719e58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.876520 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4ed6100ce34e16bf75d9e8f5e5092d', 'request_id': 'req-de51f086-b987-47cb-ab3c-59b5a8719e58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.928111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45fc48d473ac4cc8b7e3932651420fdb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.945082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45fc48d473ac4cc8b7e3932651420fdb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.976956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-655ba1cf-02b5-40f6-8f77-899331cfca5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45fc48d473ac4cc8b7e3932651420fdb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.999288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1136/2261] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:54 2022] DELETE /identity/v3/users/45fc48d473ac4cc8b7e3932651420fdb => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:55.004891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.009048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.011483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.032711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4ed6100ce34e16bf75d9e8f5e5092d', 'request_id': 'req-de51f086-b987-47cb-ab3c-59b5a8719e58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.062777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] The token cache is being invalidate because project db4ed6100ce34e16bf75d9e8f5e5092d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:55.063179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.063986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-de51f086-b987-47cb-ab3c-59b5a8719e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.066970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1126/2262] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:54 2022] DELETE /identity/v3/projects/db4ed6100ce34e16bf75d9e8f5e5092d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:55.068291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:55.069617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:55.071513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d8a1-34f6-4b6d-ae2e-6cbc0cc3e18e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.071923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d8a1-34f6-4b6d-ae2e-6cbc0cc3e18e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.072437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d8a1-34f6-4b6d-ae2e-6cbc0cc3e18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.119807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:55.141399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6bb8d8a1-34f6-4b6d-ae2e-6cbc0cc3e18e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:55.145190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6bb8d8a1-34f6-4b6d-ae2e-6cbc0cc3e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:55.254851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:55.270132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1137/2263] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:55.281813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:55.301014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:55.340983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1127/2264] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:55.363969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.367493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.372065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.464832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5730f9-f147-4201-ad47-c71f8868f00d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:55.477098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:55.481058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.481058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.481475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.508125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:55.512286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:55.557735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1128/2265] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:55.562441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:55.563591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:55.570167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:55.600333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a92814-a26e-4bcf-ba24-fa944e3bdd65 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:55.611268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1138/2266] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:55.615149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:55.615927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.617796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.618161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.619975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:55.621765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:55.623658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:55.656083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1129/2267] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:55.674347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:55.692109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493c82-1085-454a-80a1-65fcb066f781 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:55.700447 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:55.701144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.702525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.702849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.703497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:55.705461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:55.851879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1139/2268] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:55 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:55.856541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:55.867176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b130c4-4516-47e5-9329-f33632e310bc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:55.994861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:55.995525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.995806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.996083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.002372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.005226 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.143451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1130/2269] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:55 2022] POST /identity/v3/users => generated 288 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:56.144231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.167479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ded10-0218-4a71-99b9-d28350837467 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.176813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.177814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.179615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.179615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.179615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.184198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.221813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1140/2270] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:56.226266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.248563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a05e-7fc6-4f78-a75d-82f461729a55 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.269956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa9b6b-5352-4c3b-9bc7-5cc505f6c56f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.303633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1131/2271] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:56.308393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.328202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dc97a-e2cf-4d3d-8c57-f6f270bb1762 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.338294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.339611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.342630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.343276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] PATH_INFO: `/v3/projects/ac02769d1a3b41edbb870f510208201e/users/c558600671ee476aab95092b3c0b64e1/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.377536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac02769d1a3b41edbb870f510208201e, user_id=c558600671ee476aab95092b3c0b64e1, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.385775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.485247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1141/2272] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:56 2022] PUT /identity/v3/projects/ac02769d1a3b41edbb870f510208201e/users/c558600671ee476aab95092b3c0b64e1/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:56.496227 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.496763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.497231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.545280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] MFA Rules not processed for user `c558600671ee476aab95092b3c0b64e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:56.546711 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.688137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1132/2273] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:56.700227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.702863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.825769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d0cab-2337-4c94-a032-ac1a99d38fe8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.835345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.836367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.836873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.838001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.860499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.861993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50006c-5468-4d68-8461-360750ffbef2 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.874069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1142/2274] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:57.410907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.413184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:57.498552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:57.654798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0de9e-6f93-4887-b16a-adfa01c60812 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.689714 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.701406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.702054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.702547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] PATH_INFO: `/v3/projects/032e20ffecdc448eb56efdf595c67738` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.713013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=032e20ffecdc448eb56efdf595c67738)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.725235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.944609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032e20ffecdc448eb56efdf595c67738', 'request_id': 'req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:57.957402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032e20ffecdc448eb56efdf595c67738', 'request_id': 'req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:58.072841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032e20ffecdc448eb56efdf595c67738', 'request_id': 'req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:58.098238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] The token cache is being invalidate because project 032e20ffecdc448eb56efdf595c67738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:58.098919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:58.100597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d16871d6-d429-4fb1-ba1a-0d5d058bc8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:58.106988 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1133/2275] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:57 2022] DELETE /identity/v3/projects/032e20ffecdc448eb56efdf595c67738 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:58.108787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.109278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.109466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.133642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:58.158870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:58.160292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:58.250873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1143/2276] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:58.263458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.264854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.265565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.341412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:58.349437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:58.375800 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1134/2277] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:58.393176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:58.401210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd79a22-1a98-4483-b0b5-2ffd0dbd22c2 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:58.409241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:58.410251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.410806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.411404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.412287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:58.412954 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:58.415380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:58.439731 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1144/2278] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:58.449643 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:58.461861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2ff6d-298c-4e4d-afcc-b6758871e7d5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:58.484647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:58.485587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.485908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.486912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.487605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:58.492093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:58.628606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1135/2279] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:58 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:58.634857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:58.655568 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524fa2-b733-4eb6-929b-d19a097f0b02 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:58.748226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:58.749375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.749725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.750272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.753502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:58.753776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:58.911792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:58.921620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1145/2280] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:58 2022] POST /identity/v3/users => generated 285 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:58.935372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a291-bb02-4377-a3a0-d5bfbdc35c95 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:58.942448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:58.943793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.945019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.945591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.946533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:58.948780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:58.962751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1136/2281] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:58.968985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:59.000055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f25f-e7a2-4dee-88a6-9836ade8f259 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:59.011235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:59.011784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:59.012064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:59.012335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:59.013097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:59.016354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c77f4-b74c-462d-abe7-47f8b1de960d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:59.045203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:59.056886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac38588-7393-4a36-8edc-29ad528d01e3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:59.057153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1146/2282] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:59.063312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:59.063992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:59.067715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:59.067715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] PATH_INFO: `/v3/projects/c11a53bc23b24875b4dad77eeba08928/users/e6151b7020214b769e0f87c5c40024db/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:59.092722 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11a53bc23b24875b4dad77eeba08928, user_id=e6151b7020214b769e0f87c5c40024db, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:59.098814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:59.883312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1137/2283] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:59 2022] PUT /identity/v3/projects/c11a53bc23b24875b4dad77eeba08928/users/e6151b7020214b769e0f87c5c40024db/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:59.902501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:59.902501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:59.902501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:59.979005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] MFA Rules not processed for user `e6151b7020214b769e0f87c5c40024db`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:59.981373 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:00.163159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1147/2284] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:58:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:00.206251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:00.209115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:00.359437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:00.429581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3485812-48e8-4818-a83d-64b3b6a80f80 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:00.456286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:00.457030 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:00.459729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:00.460784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:00.484903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:00.500229 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc6698-09b1-4a97-b502-9dd3d5d8b3ea service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:00.523441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1138/2285] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:59:09.169051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:09.173942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:09.175850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:09.262808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aeb497-9527-45db-b02b-e7ae1056694a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:09.268583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:09.269880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.270550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.271615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.273701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:09.286488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:09.288257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.304900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1148/2286] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:10.369840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07f29-4330-4c29-a024-5d424520d05a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:10.369840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07f29-4330-4c29-a024-5d424520d05a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:10.369840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07f29-4330-4c29-a024-5d424520d05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:10.393174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-eab07f29-4330-4c29-a024-5d424520d05a None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:10.394378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eab07f29-4330-4c29-a024-5d424520d05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:10.569572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1139/2287] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:59:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:10.610821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:10.612811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:10.758261 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347a4c-bda0-43ea-b16d-ab771558606e service placement] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:10.763332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:10.764140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:10.764461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:10.764733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:10.779561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:10.781205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec85a0-b49c-4688-b9e1-c1570c03f89d service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:10.793437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1149/2288] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:59:13.111983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:13.112572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:13.112974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:13.116795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1140/2289] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:59:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:59:13.125526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf63-aba3-49ac-ace9-74ae539b15cc None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:13.139004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf63-aba3-49ac-ace9-74ae539b15cc None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:13.139004 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf63-aba3-49ac-ace9-74ae539b15cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:13.170688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-45e4cf63-aba3-49ac-ace9-74ae539b15cc None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:13.170688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-45e4cf63-aba3-49ac-ace9-74ae539b15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:13.269450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1150/2290] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:59:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:13.389251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:13.391413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:13.563636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf71661-d5d4-4d1b-93fd-c8bbc3c23230 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:13.573034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:13.573963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:13.574396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:13.574885 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] PATH_INFO: `/v3/users/c558600671ee476aab95092b3c0b64e1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:13.576322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c558600671ee476aab95092b3c0b64e1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:13.578739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:13.778014 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c558600671ee476aab95092b3c0b64e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:13.852883 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c558600671ee476aab95092b3c0b64e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:13.862682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c558600671ee476aab95092b3c0b64e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:13.863414 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c558600671ee476aab95092b3c0b64e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:13.881737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c558600671ee476aab95092b3c0b64e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:13.886461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:13.895754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1141/2291] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:13 2022] DELETE /identity/v3/users/c558600671ee476aab95092b3c0b64e1 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:13.902496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:13.903610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:13.904138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:13.940555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:13.942627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:13.959008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:14.038252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1151/2292] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:14.049069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:14.050756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:14.150254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056c242-9a7d-4c70-af41-a0e8ae96e927 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:14.158624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:14.159767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:14.160276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:14.160744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:14.173045 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:14.175274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a887-c663-4bba-842f-c801f26122aa service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:14.200435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1142/2293] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:15.061636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:15.065233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:15.098637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42483-93b1-453b-ac2b-e5174c8bad43 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:15.106348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:15.107374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.108066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.108247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] PATH_INFO: `/v3/projects/ac02769d1a3b41edbb870f510208201e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.110025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac02769d1a3b41edbb870f510208201e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:15.112362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:15.267145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac02769d1a3b41edbb870f510208201e', 'request_id': 'req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:15.273654 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac02769d1a3b41edbb870f510208201e', 'request_id': 'req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:15.340371 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac02769d1a3b41edbb870f510208201e', 'request_id': 'req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:15.355306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] The token cache is being invalidate because project ac02769d1a3b41edbb870f510208201e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:15.355722 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:15.356244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b57cdac5-4990-4caa-ad5f-c4bb54d3513b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:15.363056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1152/2294] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:15 2022] DELETE /identity/v3/projects/ac02769d1a3b41edbb870f510208201e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:15.363293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.363784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.364001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.391745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:15.394415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:15.495941 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1143/2295] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:15.504433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.504782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.505066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.532989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:15.534387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:15.554355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:15.562708 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1153/2296] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:15.568605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032414c0-b167-4d92-aaae-35bf29c6f85a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:15.574911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:15.575815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.576458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.576900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.577760 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:59:15.578244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:15.580113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:15.598015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1144/2297] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:59:15.599465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:15.610477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f3607-74f3-4029-a5dc-f0f382216f7a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:15.616155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:15.617090 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.618059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.618499 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.619506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:15.621885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:15.759173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1154/2298] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:15 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:15.760266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:15.775327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d7192-7cec-44da-8915-ade3c0e6a727 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:15.873477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:15.874008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.874266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.874547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.875222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:15.877252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:15.977699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1145/2299] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:15 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:15.979713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:15.993248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408613e1-ff82-4a77-827f-d826f7545fce admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:16.000684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:16.000684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:16.000684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:16.000684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:16.001289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:16.004968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:16.022710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1155/2300] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:16.025378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:16.045778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdb22-2b23-4630-836c-9fb55b9e2b74 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:16.054750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:16.056477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:16.057046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:16.057605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:16.059024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:16.062367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bfc16-b504-4508-b1cc-f99aa2e1a39d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:16.078559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1146/2301] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:59:16.079861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:16.091789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565054-76ab-4b48-a8b2-b78be67fd64c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:16.097148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:16.098732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:16.099232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:16.099690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] PATH_INFO: `/v3/projects/ea9bce968d2a47b58bcc2acc5db77025/users/00b6c965bcd7439795e02efa5c0cf34f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:16.118972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9bce968d2a47b58bcc2acc5db77025, user_id=00b6c965bcd7439795e02efa5c0cf34f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:16.121788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:16.197018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1156/2302] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:16 2022] PUT /identity/v3/projects/ea9bce968d2a47b58bcc2acc5db77025/users/00b6c965bcd7439795e02efa5c0cf34f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:16.207726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa67421-fb7c-4e3c-9bed-2d0cf482f65e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:16.208004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa67421-fb7c-4e3c-9bed-2d0cf482f65e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:16.208255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa67421-fb7c-4e3c-9bed-2d0cf482f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:16.231077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:16.241855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3fa67421-fb7c-4e3c-9bed-2d0cf482f65e None None] MFA Rules not processed for user `00b6c965bcd7439795e02efa5c0cf34f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:16.243357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3fa67421-fb7c-4e3c-9bed-2d0cf482f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:16.343559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1147/2303] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:16.360040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:16.362688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:16.407162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:16.446710 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f1b22-f99e-4b95-8780-0bb69acf7343 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:16.447905 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:16.448583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:16.448870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:16.449146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:16.462379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:16.465120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:16.478913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1157/2304] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:18.657023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d382-a7e8-4cb7-aff1-f08e71b5c3a9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:18.657644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d382-a7e8-4cb7-aff1-f08e71b5c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:18.657959 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d382-a7e8-4cb7-aff1-f08e71b5c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:18.708317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-dc41d382-a7e8-4cb7-aff1-f08e71b5c3a9 None None] MFA Rules not processed for user `00b6c965bcd7439795e02efa5c0cf34f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:18.710706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dc41d382-a7e8-4cb7-aff1-f08e71b5c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:18.723236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1148/2305] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:20.405808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:20.408002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:20.501028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e59e1-3c40-4a18-8788-f91d256471b1 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:20.506508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:20.507288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:20.507781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:20.508718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:20.511613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:20.651535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:20.653332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289193ef-bffb-427c-b34e-36fcbe64b961 service glance] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:20.666476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1158/2306] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:20.939687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:20.943874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:20.943874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:20.943874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1149/2307] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:59:20.951110 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb6e9-dc9a-42bc-b64d-c844dc17fc05 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:20.951745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb6e9-dc9a-42bc-b64d-c844dc17fc05 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:20.952230 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb6e9-dc9a-42bc-b64d-c844dc17fc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:20.984680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-825fb6e9-dc9a-42bc-b64d-c844dc17fc05 None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:20.986843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-825fb6e9-dc9a-42bc-b64d-c844dc17fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:21.109802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1159/2308] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:59:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:21.126769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:21.130453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:21.149286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713bb0-51a2-4ef9-8f3e-482e476273a3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:21.153642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:21.154283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:21.154803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:21.155231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:21.170068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:21.196432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda0bd6-cea6-415a-9250-534a772e0aff service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:21.224456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1150/2309] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:21.354563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:21.354900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:21.355180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:21.356551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1160/2310] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 08:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:59:21.367150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659185-9c86-46d0-b210-45b109a26511 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:21.367807 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659185-9c86-46d0-b210-45b109a26511 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:21.368386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd659185-9c86-46d0-b210-45b109a26511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:21.423109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bd659185-9c86-46d0-b210-45b109a26511 None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:21.426359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bd659185-9c86-46d0-b210-45b109a26511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:21.441056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1151/2311] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 08:59:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:21.454431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:21.456857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:21.476990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b93b4-528f-475b-8efc-c9672b9b1d91 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:21.481322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:21.481935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:21.482219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:21.482489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:21.504248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:21.513147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:21.539647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1161/2312] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:31.392286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=40 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:31.523509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:40.189511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dca0-69fb-44f2-8a92-4f4aea7e9c4f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:40.190021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dca0-69fb-44f2-8a92-4f4aea7e9c4f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:40.190117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94dca0-69fb-44f2-8a92-4f4aea7e9c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:40.210741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cf94dca0-69fb-44f2-8a92-4f4aea7e9c4f None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:40.212051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cf94dca0-69fb-44f2-8a92-4f4aea7e9c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:40.270430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1152/2313] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 08:59:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:40.283533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:40.284977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:40.361111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe94d04-561f-4569-addb-892137158a31 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:40.366026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:40.367244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:40.367527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:40.367796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:40.376672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:40.378474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:40.388066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1162/2314] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:59:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 08:59:40.997614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:40.997809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:40.998241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:41.017898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:41.019211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:41.027536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1153/2315] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 08:59:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:41.041557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:41.052514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cd44-0f8f-4872-8a73-3ba20f869b71 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:41.057973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:41.058636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:41.058940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:41.059212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:41.067592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:41.069383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b763a-e973-45d8-8e8f-3fddd401be65 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:41.079596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1163/2316] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 08:59:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:59:42.243742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:42.247090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:42.376865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019075-f6c7-4aec-951f-5b5d1941f00c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:42.383755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:42.384767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:42.385309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:42.385769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] PATH_INFO: `/v3/users/e6151b7020214b769e0f87c5c40024db` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:42.387242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6151b7020214b769e0f87c5c40024db)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:42.389605 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:42.648756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6151b7020214b769e0f87c5c40024db'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:42.737477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6151b7020214b769e0f87c5c40024db'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:42.746823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6151b7020214b769e0f87c5c40024db'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:42.747687 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6151b7020214b769e0f87c5c40024db'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:42.761162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6151b7020214b769e0f87c5c40024db'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:42.767239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1154/2317] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:42 2022] DELETE /identity/v3/users/e6151b7020214b769e0f87c5c40024db => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:42.771727 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:42.772222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:42.776605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:42.791218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:42.792402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:42.878376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1164/2318] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:42.886216 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:42.887792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:42.910420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:42.949401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b9cbd-30d8-42ed-af6a-bf1c5bd10af7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:42.953195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:42.953813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:42.954096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:42.954368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:42.966380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:42.968104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:42.982979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1155/2319] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:44.070308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:44.073458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:44.102530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9524b0-f3ff-4472-be78-e20fd54b3a6c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:44.108761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:44.109841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:44.110381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:44.111609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] PATH_INFO: `/v3/projects/c11a53bc23b24875b4dad77eeba08928` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:44.114337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11a53bc23b24875b4dad77eeba08928)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:44.117854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:44.374586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11a53bc23b24875b4dad77eeba08928', 'request_id': 'req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:44.382445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11a53bc23b24875b4dad77eeba08928', 'request_id': 'req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:44.511376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11a53bc23b24875b4dad77eeba08928', 'request_id': 'req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:44.544316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:44.558892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] The token cache is being invalidate because project c11a53bc23b24875b4dad77eeba08928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:44.559195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:44.561543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:44.578976 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1165/2320] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:44 2022] DELETE /identity/v3/projects/c11a53bc23b24875b4dad77eeba08928 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:45.593369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:45.596504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:45.613916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d4770-0401-4e4c-884f-04515a2ecb64 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:45.688111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:45.689195 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:45.689727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:45.690355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] PATH_INFO: `/v3/users/f7f0342a701f492a9de96588c1569fac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:45.692091 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f0342a701f492a9de96588c1569fac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:45.707672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:45.961829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f0342a701f492a9de96588c1569fac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.041745 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f0342a701f492a9de96588c1569fac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.054480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f0342a701f492a9de96588c1569fac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.055662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f0342a701f492a9de96588c1569fac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.070976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-174c99b5-cc5f-45ea-9dfd-9affdfd404bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f0342a701f492a9de96588c1569fac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.077967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1156/2321] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:45 2022] DELETE /identity/v3/users/f7f0342a701f492a9de96588c1569fac => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:46.448820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:46.450385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:46.463381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd68d1-cb35-4480-9ca4-08d1b9a25ed4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:46.541649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:46.542304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:46.542589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:46.542916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] PATH_INFO: `/v3/projects/fadf4161dc28487487541cefeae7a220` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:46.544077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fadf4161dc28487487541cefeae7a220)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:46.546007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:46.660614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fadf4161dc28487487541cefeae7a220', 'request_id': 'req-32908fcd-2a19-4546-8c2c-6d72341e79a5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.667585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fadf4161dc28487487541cefeae7a220', 'request_id': 'req-32908fcd-2a19-4546-8c2c-6d72341e79a5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.780121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fadf4161dc28487487541cefeae7a220', 'request_id': 'req-32908fcd-2a19-4546-8c2c-6d72341e79a5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.794325 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] The token cache is being invalidate because project fadf4161dc28487487541cefeae7a220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:46.796231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.796231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-32908fcd-2a19-4546-8c2c-6d72341e79a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:46.796934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1166/2322] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:46 2022] DELETE /identity/v3/projects/fadf4161dc28487487541cefeae7a220 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:46.801985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:46.802523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:46.802629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:46.823278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:46.824484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:46.865126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:46.911686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1157/2323] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:46.920445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:46.920748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:46.921000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:46.943299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:46.946013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:46.954556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1167/2324] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:46.958692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:46.969604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e0df-47b9-4f20-a5ec-8569588ec146 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:46.974481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:46.975135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:46.975442 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:46.975742 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:46.976416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:59:46.976720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:46.978343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:46.991351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1158/2325] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 08:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:59:46.996894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.010608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72201e-826f-4c59-9739-66687e4d6824 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.021754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.023260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.023615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.023896 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.024539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.028864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.126406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1168/2326] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:46 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:47.130419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.143322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73352-15cd-43d2-b0eb-56654a7d03b0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.239837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.241071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.241832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.241832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.243479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.246334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.340666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1159/2327] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:47 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:47.347199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.358637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea102b-c0b9-46aa-9032-f6fa078fd3e2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.363570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.366188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.366188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.366188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.366188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.367197 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.379186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1169/2328] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:47.384335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.397280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d684bfe-1156-42d1-8f21-3c4974d60ec6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.403311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.404247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.404784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.405223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.406215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.407978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6b4cf-51e8-4e8d-9234-2726731938c6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.420158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1160/2329] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:47.424738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.435839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcc61f-0f68-4062-b7f9-879c674e59b9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.440521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.441185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.441468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.441741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] PATH_INFO: `/v3/projects/ad240f302f344e1d9f609ce533ecc72e/users/3ec969e2e4f64c608d75965db8e441a4/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.458372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad240f302f344e1d9f609ce533ecc72e, user_id=3ec969e2e4f64c608d75965db8e441a4, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.461735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.548420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1170/2330] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:47 2022] PUT /identity/v3/projects/ad240f302f344e1d9f609ce533ecc72e/users/3ec969e2e4f64c608d75965db8e441a4/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:47.564543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7b16-0da3-47ac-99b5-6fdbacdd4872 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.564938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7b16-0da3-47ac-99b5-6fdbacdd4872 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.565314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c7b16-0da3-47ac-99b5-6fdbacdd4872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.601543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-559c7b16-0da3-47ac-99b5-6fdbacdd4872 None None] MFA Rules not processed for user `3ec969e2e4f64c608d75965db8e441a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:47.602811 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-559c7b16-0da3-47ac-99b5-6fdbacdd4872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:47.691512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1161/2331] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:47.704105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:47.705964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:47.743304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:47.769956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c74e62-f0f0-4c10-b1ca-30be11e30f90 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:47.774292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:47.775042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:47.775339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:47.775653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:47.785505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:47.787009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:47.800144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1171/2332] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:57.687579 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:57.865191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=14 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:10.541516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0696f-fc25-4dcd-8dd9-94357f024970 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:10.541516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0696f-fc25-4dcd-8dd9-94357f024970 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:10.541516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0696f-fc25-4dcd-8dd9-94357f024970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:10.559235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cae0696f-fc25-4dcd-8dd9-94357f024970 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:10.560519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cae0696f-fc25-4dcd-8dd9-94357f024970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:10.640196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1162/2333] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:10.671132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:10.671132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:10.739775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875415-e235-4782-ad90-ffdeae01c1f7 service glance] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:10.745064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:10.745681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:10.745961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:10.752495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:10.757713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:10.759603 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:10.768953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1172/2334] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:11.701905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613cae6-5166-42bf-b2fc-b98c8c0ca99e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:11.702379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613cae6-5166-42bf-b2fc-b98c8c0ca99e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:11.702473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613cae6-5166-42bf-b2fc-b98c8c0ca99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:11.721923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8613cae6-5166-42bf-b2fc-b98c8c0ca99e None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:11.723159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8613cae6-5166-42bf-b2fc-b98c8c0ca99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:11.730298 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1163/2335] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 09:00:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:11.743433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:11.754133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bd7-1bd6-4f28-8872-38ff15c506c3 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:11.759987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:11.760627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:11.760912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:11.761187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:11.769678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:11.771952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:11.780440 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1173/2336] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:12.476560 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5eb0-9b93-4d7a-9ca9-ccb79dfe41a7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:12.476849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5eb0-9b93-4d7a-9ca9-ccb79dfe41a7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:12.477125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5eb0-9b93-4d7a-9ca9-ccb79dfe41a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:12.497204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b5ad5eb0-9b93-4d7a-9ca9-ccb79dfe41a7 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:12.498471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b5ad5eb0-9b93-4d7a-9ca9-ccb79dfe41a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:12.506252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1164/2337] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:12.518497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:12.529925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ce73-8fca-428b-93b8-d0bd39d88b53 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:12.536103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:12.536251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:12.536549 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:12.537484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:12.546558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:12.548194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:12.557317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1174/2338] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:13.141608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f8a9-e53d-4bc1-8997-6b39b804d12a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:13.142071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f8a9-e53d-4bc1-8997-6b39b804d12a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:13.142164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f8a9-e53d-4bc1-8997-6b39b804d12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:13.162021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2458f8a9-e53d-4bc1-8997-6b39b804d12a None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:13.163709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2458f8a9-e53d-4bc1-8997-6b39b804d12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:13.171210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1165/2339] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 09:00:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:13.183491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:13.194815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad40ccd-48dd-4c2f-8365-8f6a62ec4956 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:13.200675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:13.201295 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:13.201598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:13.201884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:13.212697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:13.215296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:13.223782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1175/2340] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:18.656544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:18.663511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:18.663511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:18.684211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:18.686318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:18.699607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1166/2341] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 09:00:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:18.716942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:18.730060 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66dba-1a9a-494e-aace-a5970dc21ba9 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:18.735916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:18.736422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:18.736702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:18.736975 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:18.747782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:18.749899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6790c1f-af1d-48ff-b251-0c2f80dab23c service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:18.761231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1176/2342] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:19.569849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:19.573497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:19.680485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543c632-440d-48c8-ac71-380b3c5b2e7d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:19.686017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:19.686564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:19.686882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:19.687148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] PATH_INFO: `/v3/users/00b6c965bcd7439795e02efa5c0cf34f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:19.713614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b6c965bcd7439795e02efa5c0cf34f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:19.713614 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:20.024871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b6c965bcd7439795e02efa5c0cf34f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.126194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b6c965bcd7439795e02efa5c0cf34f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.140428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b6c965bcd7439795e02efa5c0cf34f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.141731 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b6c965bcd7439795e02efa5c0cf34f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.157070 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b6c965bcd7439795e02efa5c0cf34f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.159334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:20.165392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1167/2343] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:19 2022] DELETE /identity/v3/users/00b6c965bcd7439795e02efa5c0cf34f => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:20.170575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:20.171136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:20.171355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:20.194689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:20.194689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:20.280804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1177/2344] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:20.295029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:20.299407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:20.375956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af82f-a1a0-492a-9d2c-8883d35f2c0c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:20.381059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:20.381671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:20.381948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:20.382228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:20.392554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:20.394021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b946c3-00ba-453b-bae4-868df3e62f94 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:20.404695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1168/2345] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:20.763081 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:20.765219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:20.788876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7fe0-4471-40ca-b725-a91d44fedb8c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:20.792538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:20.793203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:20.793482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:20.793764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] PATH_INFO: `/v3/projects/ea9bce968d2a47b58bcc2acc5db77025` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:20.795066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9bce968d2a47b58bcc2acc5db77025)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:20.796999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:20.912169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:20.932246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9bce968d2a47b58bcc2acc5db77025', 'request_id': 'req-e266c330-e9ff-43d3-851f-a7d9395b878a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:20.940581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9bce968d2a47b58bcc2acc5db77025', 'request_id': 'req-e266c330-e9ff-43d3-851f-a7d9395b878a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.099970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9bce968d2a47b58bcc2acc5db77025', 'request_id': 'req-e266c330-e9ff-43d3-851f-a7d9395b878a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.113420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] The token cache is being invalidate because project ea9bce968d2a47b58bcc2acc5db77025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:21.113713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.114375 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e266c330-e9ff-43d3-851f-a7d9395b878a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.115903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1178/2346] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:20 2022] DELETE /identity/v3/projects/ea9bce968d2a47b58bcc2acc5db77025 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:21.121221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.121767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.121767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.143946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.145297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.212788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1169/2347] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.280311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.280912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.280912 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.301589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.302902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.311307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1179/2348] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.315703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.326418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3b45d-7ff0-426a-87f8-ee47fd80a4d3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.331463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.332110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.332401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.332682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.334008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:21.334313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.335972 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.349019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1170/2349] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:00:21.353422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.363429 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5cae2-5bb1-4c55-b716-d03a5d7c9fd1 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.369444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.370129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.370436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.370901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.371543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.373478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.467768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1180/2350] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:21.472598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.485680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a37cc-7a66-4f11-9b59-136b9226cd05 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.556501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.557126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.557408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.557684 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.558394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.560494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.681937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1171/2351] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:21.686364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.697808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bbb3c-d76a-4775-9f74-826aaa741f5f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.703271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.707263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.707263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.707263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.707263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.707263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.718847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1181/2352] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:21.725000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.742836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188113c5-3790-410b-a497-ff4581fa0a28 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.747974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.748628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.749092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.749372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.750034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.752027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc4f60-6ac5-4fe3-9fa7-6fd7df7a684d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.764814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1172/2353] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:21.769721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.779843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72309d-ac32-4d7a-abad-f598ef2d1cb6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.784483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.785143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.785419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.785686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] PATH_INFO: `/v3/projects/065b8b1394f24ce1973119ee8cd6bd2c/users/a35fcaecf09947e78813f96ec93e6c0a/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.801689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065b8b1394f24ce1973119ee8cd6bd2c, user_id=a35fcaecf09947e78813f96ec93e6c0a, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.804902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.901448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1182/2354] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:21 2022] PUT /identity/v3/projects/065b8b1394f24ce1973119ee8cd6bd2c/users/a35fcaecf09947e78813f96ec93e6c0a/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:21.909395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a8c1-9bcf-4c45-b904-aea3811864b1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.909878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a8c1-9bcf-4c45-b904-aea3811864b1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.910352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a8c1-9bcf-4c45-b904-aea3811864b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.935570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:21.944234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-afb2a8c1-9bcf-4c45-b904-aea3811864b1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.946154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-afb2a8c1-9bcf-4c45-b904-aea3811864b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:22.059336 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1173/2355] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:22.071761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:22.074677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:22.166939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36a900-b49e-4e16-b3f8-d5dfbccc5a70 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:22.171610 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:22.172639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:22.173126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:22.173594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:22.187321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:22.190056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:22.210213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1183/2356] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:28.393284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.393855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.394246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.472756 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] MFA Rules not processed for user `a35fcaecf09947e78813f96ec93e6c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:28.474432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.580039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1174/2357] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:28.585415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.600202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb4315-d76f-4870-b54c-b2efb54a41d0 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.607768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.608404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.608691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.608968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.609705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:28.612017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:28.720744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1184/2358] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:28 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:28.722031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.734429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b530a-98a5-41ca-9a85-739bcd9eab97 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.812066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.812802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.813165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.813448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.814177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:28.816639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:28.923829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1175/2359] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:28 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:28.927161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.940743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609655-fadb-486e-b8bd-a71afba6b017 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.946535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.947210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.947496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.947805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.948665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:28.950351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:28.963020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1185/2360] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:28.968565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.984688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ea886-8f07-4245-81b5-807a151f3c7e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.985874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.986555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.986864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.987142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] PATH_INFO: `/v3/projects/5b1ec073f8634ba2927ea52b735f653c/users/e13368f0a82b42039f5e9a965d6afcab/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:29.019013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1ec073f8634ba2927ea52b735f653c, user_id=e13368f0a82b42039f5e9a965d6afcab, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:29.022795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:29.177109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1176/2361] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:28 2022] PUT /identity/v3/projects/5b1ec073f8634ba2927ea52b735f653c/users/e13368f0a82b42039f5e9a965d6afcab/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:29.181817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:29.193030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93319a17-0478-476a-8e0a-c099b61a94f1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:29.221561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:29.259875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:29.260596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:29.261032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:29.261369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:29.262087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:29.263935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:29.278315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1186/2362] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:29.281034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:29.291270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:29.297840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8510a6-0b3f-42cf-9ad3-f6d596d2c118 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:29.311116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:29.312954 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:29.313581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:29.314230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] PATH_INFO: `/v3/projects/5b1ec073f8634ba2927ea52b735f653c/users/e13368f0a82b42039f5e9a965d6afcab/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:29.317437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1ec073f8634ba2927ea52b735f653c, user_id=e13368f0a82b42039f5e9a965d6afcab, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:29.320535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245abd1f-eb15-4f34-95e6-f06f832da076 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:29.336878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1177/2363] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:29 2022] PUT /identity/v3/projects/5b1ec073f8634ba2927ea52b735f653c/users/e13368f0a82b42039f5e9a965d6afcab/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:33.452911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:33.457878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:33.473042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c16f0-9dc5-478c-a448-1bc6803f3543 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:33.566251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:33.566959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:33.567734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:33.568048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] PATH_INFO: `/v3/users/3ec969e2e4f64c608d75965db8e441a4` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:33.569276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec969e2e4f64c608d75965db8e441a4)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:33.572479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:33.983973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec969e2e4f64c608d75965db8e441a4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:33.985510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec969e2e4f64c608d75965db8e441a4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.103106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec969e2e4f64c608d75965db8e441a4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.115689 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec969e2e4f64c608d75965db8e441a4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.145507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec969e2e4f64c608d75965db8e441a4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.158318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1187/2364] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:33 2022] DELETE /identity/v3/users/3ec969e2e4f64c608d75965db8e441a4 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:34.159919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:34.160744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:34.161181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:34.187259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:34.188485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:34.321985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1178/2365] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:34.326273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:34.328107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:34.445387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:34.465591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02fb0-4a49-4964-b943-3cdded4ea486 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:34.470855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:34.471504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:34.471916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:34.472190 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:34.513986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:34.513986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede322d-908d-4c4b-9ba3-44bdfa897d65 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:34.515850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1188/2366] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:35.412879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:35.415142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.430837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b605782-2831-4ca5-a0fb-8d14dfec106e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:35.440142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:35.442821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.442821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.442821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] PATH_INFO: `/v3/projects/ad240f302f344e1d9f609ce533ecc72e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.442821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad240f302f344e1d9f609ce533ecc72e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:35.445662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:35.613142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad240f302f344e1d9f609ce533ecc72e', 'request_id': 'req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.621485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad240f302f344e1d9f609ce533ecc72e', 'request_id': 'req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.721914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad240f302f344e1d9f609ce533ecc72e', 'request_id': 'req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.758845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] The token cache is being invalidate because project ad240f302f344e1d9f609ce533ecc72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:35.759199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.760314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-50a9a3c3-2235-4b4f-9e2f-4049ce2aed33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.763228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1179/2367] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:35 2022] DELETE /identity/v3/projects/ad240f302f344e1d9f609ce533ecc72e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:35.768978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.769895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.770399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.800744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:35.802214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.898036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1189/2368] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:35.899290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.899675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.900181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.923916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:35.925302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.940159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1180/2369] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:35.943715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:35.966444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3245-177a-41c1-b32e-0e16c1627e4e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:35.992696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:35.996035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.997091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.997859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.999477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:36.000328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.006269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.042580 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1190/2370] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:00:36.042924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.054999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b1be0-3bd0-4f4b-870e-a6db32355d2a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.064162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.064910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.065252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.065589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.066338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.069142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.227532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1181/2371] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/projects => generated 389 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:36.231822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.245874 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bca8f-7136-4f7e-95f2-2348c7c73f8c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.410203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.411266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.412386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.412878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.413910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.417740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.621877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1191/2372] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/users => generated 295 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:36.624311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.638677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d7e2-eb35-4a48-a794-cdf4389fde7b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.645640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.646691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.647178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.648285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.649164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.651937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.670512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.676228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1182/2373] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:36.683464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82856-4526-4096-99f2-4abec48820d1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.691442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.692299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.692630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.692964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.694854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.697172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d0492-f241-44fd-852e-95cfbafead2b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.722259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.735423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1192/2374] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:36.737488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d0e9d-7b32-48db-8e2a-087879f56b19 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.750158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.751237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.751650 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.752151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] PATH_INFO: `/v3/projects/b0891dcdbf6543f8b5fdf2206fefb713/users/9ca18b43172b4e39a378a0462e46d23f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.776107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0891dcdbf6543f8b5fdf2206fefb713, user_id=9ca18b43172b4e39a378a0462e46d23f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.784133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.893974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1183/2375] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:36 2022] PUT /identity/v3/projects/b0891dcdbf6543f8b5fdf2206fefb713/users/9ca18b43172b4e39a378a0462e46d23f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:36.905163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25691-badf-49c1-919f-938c7022e69b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.905769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25691-badf-49c1-919f-938c7022e69b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.905769 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25691-badf-49c1-919f-938c7022e69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.950407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-31d25691-badf-49c1-919f-938c7022e69b None None] MFA Rules not processed for user `9ca18b43172b4e39a378a0462e46d23f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:36.950736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-31d25691-badf-49c1-919f-938c7022e69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.036505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:37.079269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1193/2376] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:37.091561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.093424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.107730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:37.179748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54e201-87d6-4a85-bc53-352caefa8b4e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.184539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.186231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.186617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.187029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.211067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.214239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.229524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1184/2377] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:38.816471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b851-1335-486e-ac0f-d026c3286846 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:38.817072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b851-1335-486e-ac0f-d026c3286846 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:38.817072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b851-1335-486e-ac0f-d026c3286846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:38.868732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b377b851-1335-486e-ac0f-d026c3286846 None None] MFA Rules not processed for user `e13368f0a82b42039f5e9a965d6afcab`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:38.870868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b377b851-1335-486e-ac0f-d026c3286846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:39.008902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1194/2378] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:39.033967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:39.037271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:39.148163 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940363d6-b391-4c69-8776-59f850dfc220 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:39.155591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:39.156367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:39.156802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:39.157259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:39.159629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:39.237678 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:39.242583 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e999d1-e212-4674-adb0-3efd46021ba9 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:39.249852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1185/2379] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:41.552358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da6dd-1676-4553-baae-f6776d8c0f81 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:41.553014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da6dd-1676-4553-baae-f6776d8c0f81 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:41.553014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5da6dd-1676-4553-baae-f6776d8c0f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:41.577512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-7e5da6dd-1676-4553-baae-f6776d8c0f81 None None] MFA Rules not processed for user `3f9025a5d28b47d8bc36f131e908382f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:41.578987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7e5da6dd-1676-4553-baae-f6776d8c0f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:41.657703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1195/2380] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:43.860436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c474-97cb-432c-96ec-d15cef75c1aa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:43.861702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c474-97cb-432c-96ec-d15cef75c1aa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:43.862870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c474-97cb-432c-96ec-d15cef75c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:43.918765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c114c474-97cb-432c-96ec-d15cef75c1aa None None] MFA Rules not processed for user `a35fcaecf09947e78813f96ec93e6c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:43.928924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c114c474-97cb-432c-96ec-d15cef75c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:43.947447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1186/2381] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:51.649935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=68 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:53.900773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:54.092636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.093207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.093743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.140512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] MFA Rules not processed for user `a62220151af547aa9dbeb84f8e00c045`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:54.143357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:54.243496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1196/2382] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:01:00.747153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593c002-14d6-4209-81f0-83aad7ac6324 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:00.748089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593c002-14d6-4209-81f0-83aad7ac6324 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:00.748089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593c002-14d6-4209-81f0-83aad7ac6324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:00.773217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6593c002-14d6-4209-81f0-83aad7ac6324 None None] MFA Rules not processed for user `5171bf0514c3474aae358a9d7cfaf538`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:00.775488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6593c002-14d6-4209-81f0-83aad7ac6324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:00.881700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1187/2383] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:01:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:03.445254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:03.571891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502db4ed-d8cb-47b6-9978-2b14091181ca None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:03.578814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:03.579672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:03.579997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:03.587936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] PATH_INFO: `/v3/users/9ca18b43172b4e39a378a0462e46d23f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:03.587936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca18b43172b4e39a378a0462e46d23f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:03.587936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:03.904967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca18b43172b4e39a378a0462e46d23f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.905693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca18b43172b4e39a378a0462e46d23f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:04.012221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca18b43172b4e39a378a0462e46d23f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:04.026317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca18b43172b4e39a378a0462e46d23f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:04.047326 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca18b43172b4e39a378a0462e46d23f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:04.057506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1197/2384] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:01:03 2022] DELETE /identity/v3/users/9ca18b43172b4e39a378a0462e46d23f => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:01:04.063460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:04.067020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:04.069182 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:04.091793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:04.093249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:04.221964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1188/2385] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:04.238916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:04.242687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:04.333157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:04.385496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba407b-003f-4e3e-b3c6-d3d505f5d158 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:04.395815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:04.396883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:04.397415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:04.398125 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:04.423538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:04.426476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:04.447360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1198/2386] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:05.294198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:05.296780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.318596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b256b-7c5a-4a63-b03a-3a2a97446871 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:05.327326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:05.328264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:05.328717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:05.329107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] PATH_INFO: `/v3/projects/b0891dcdbf6543f8b5fdf2206fefb713` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:05.341995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0891dcdbf6543f8b5fdf2206fefb713)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:05.345703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:05.388946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:05.390523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.482229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe39ca-c718-4284-84c5-0af8ff6046a7 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:05.492613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:05.493275 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:05.493536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:05.493809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:05.497423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.569681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0891dcdbf6543f8b5fdf2206fefb713', 'request_id': 'req-4e292ee9-6737-40a2-ad94-09db23167ddc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:05.583726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0891dcdbf6543f8b5fdf2206fefb713', 'request_id': 'req-4e292ee9-6737-40a2-ad94-09db23167ddc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:05.670634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0891dcdbf6543f8b5fdf2206fefb713', 'request_id': 'req-4e292ee9-6737-40a2-ad94-09db23167ddc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:05.710018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:05.712752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] The token cache is being invalidate because project b0891dcdbf6543f8b5fdf2206fefb713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:01:05.712964 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97ca67-9e6b-4c94-8111-1552bce93f2d service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:05.713160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:05.718161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:05.723535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1189/2387] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:01:05 2022] DELETE /identity/v3/projects/b0891dcdbf6543f8b5fdf2206fefb713 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:05.751691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1199/2388] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:08.011894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:08.013728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:08.080257 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:08.135389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e292ee9-6737-40a2-ad94-09db23167ddc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:08.143888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:08.145613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:08.146362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:08.147044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:08.150964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:08.264506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:08.266397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:08.318790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1190/2389] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:09.420148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9acc1-964e-45fa-b465-f83ca2c0b809 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:09.421012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9acc1-964e-45fa-b465-f83ca2c0b809 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:09.421012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9acc1-964e-45fa-b465-f83ca2c0b809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:09.446703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-39f9acc1-964e-45fa-b465-f83ca2c0b809 None None] MFA Rules not processed for user `f1381dae2b774b48a590e21b3d30dbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:09.448015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-39f9acc1-964e-45fa-b465-f83ca2c0b809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:09.570704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1200/2390] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:01:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:18.715945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=41 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:19.554838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:21.158713 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:21.160531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:21.178082 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5f48f-1f05-419b-90b8-94b03562f98a service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:21.184861 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:21.187167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:21.187167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:21.187167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:21.188051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:21.265119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:21.267531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:21.283893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1191/2391] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:26.896073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:26.897571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:26.897571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:26.920466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] MFA Rules not processed for user `2cd91f6976174fbf9692e0973435a4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:26.922389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:26.932351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1201/2392] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:01:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:01:27.312698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:27.322947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e04cc5-05a8-427d-b965-b0375d52136c service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:27.326964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:27.327677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:27.327993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:27.328292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:27.330504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:27.341880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:27.343396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:27.353057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1192/2393] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:34.884200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:34.885709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:34.897601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e79069-938e-4ff7-aa0d-ddc4b65152b6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:34.901433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:34.907174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:34.907174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:34.907174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:34.907174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:34.984095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:34.985475 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:34.997917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1202/2394] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:37.349120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=34 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:44.479905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:44.502531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4381409-00a5-4a10-82b3-4819ddd6b48d service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:44.509962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:44.511545 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:44.513072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:44.513072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:44.515475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:44.533741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:44.537962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:44.559342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1193/2395] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:44.998709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=33 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:49.823795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:49.825343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:49.841112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84f9c8-810d-4612-b131-64965fbf3dac service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:49.846743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:49.847873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:49.848155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:49.848433 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:49.850163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:49.862778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:49.865031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:49.876098 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1203/2396] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:50.985002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:50.987575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:51.091525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d05fa7-32ea-4905-a8ce-2e5c1408f5cb service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:51.096031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:51.096764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:51.097157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:51.098211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:51.100203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:51.171105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:51.172626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:51.183767 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1194/2397] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:57.745199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:57.745199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0602b-992b-4995-9e79-45b4a7c6558c service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:57.745917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:57.746039 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:57.746259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:57.746533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:57.748321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:57.814313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:57.816024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:57.825133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1204/2398] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:01.178853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=54 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:07.821080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=34 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:15.605784 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:15.624564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d74-ea7c-48dc-9271-c3bdb075a75e service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:15.637380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:15.638732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:15.639236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:15.640007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:15.644358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:15.664911 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:15.667882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:15.687801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1195/2399] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:19.496490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:19.498628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:19.571313 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6680e-730a-4f34-b708-a1140cb1a7c2 service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:19.576932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:19.577093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:19.577902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:19.578245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:19.580140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:19.593111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:19.594902 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:19.608941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1205/2400] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:19.689066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:19.691183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:19.779125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608ce61-69b2-4ab2-90ae-ed31889a8139 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:19.786984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:19.787812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:19.788142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:19.788563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] PATH_INFO: `/v3/users/a62220151af547aa9dbeb84f8e00c045` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:19.789727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62220151af547aa9dbeb84f8e00c045)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:19.791737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:19.984361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:19.994853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25293151-a744-4ac1-b16c-4426fb9ee16f service placement] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:19.998533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:19.999516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:19.999782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:20.000051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:20.001937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:20.015791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:20.017512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:20.028875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1206/2401] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:02:20.084930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62220151af547aa9dbeb84f8e00c045'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:20.159290 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62220151af547aa9dbeb84f8e00c045'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:20.169112 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62220151af547aa9dbeb84f8e00c045'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:20.169853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62220151af547aa9dbeb84f8e00c045'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:20.199940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd7151c9-e060-4d26-aadb-97d5c385c365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62220151af547aa9dbeb84f8e00c045'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:20.209527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1196/2402] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:02:19 2022] DELETE /identity/v3/users/a62220151af547aa9dbeb84f8e00c045 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:02:20.787018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:20.797481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fab104-faa5-487a-bf7f-16f16e9e16c8 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:20.870681 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:20.871522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:20.871796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:20.872337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] PATH_INFO: `/v3/projects/e3fc853baefe4aa6b105874207810d01` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:20.873670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fc853baefe4aa6b105874207810d01)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:20.876048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.043191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fc853baefe4aa6b105874207810d01', 'request_id': 'req-e4d6ca5f-0c13-4896-9221-db38d31076b9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:21.051413 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fc853baefe4aa6b105874207810d01', 'request_id': 'req-e4d6ca5f-0c13-4896-9221-db38d31076b9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:21.155351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fc853baefe4aa6b105874207810d01', 'request_id': 'req-e4d6ca5f-0c13-4896-9221-db38d31076b9'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:21.174529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] The token cache is being invalidate because project e3fc853baefe4aa6b105874207810d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:02:21.175013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:21.175740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-e4d6ca5f-0c13-4896-9221-db38d31076b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:21.182808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1207/2403] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:02:20 2022] DELETE /identity/v3/projects/e3fc853baefe4aa6b105874207810d01 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:21.186719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.188334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.189106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.214476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:21.215325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:21.287205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1197/2404] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:21.295396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.295683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.296016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.311014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:21.318348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:21.320345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:21.330688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1208/2405] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:02:21.336066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.346243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a6d97-4c97-47c0-aa40-0aa5e4bebca2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.352549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.355033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.355033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.355033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.355033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:02:21.355033 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.356609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.358204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:21.372891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1198/2406] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:02:21.379281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.398108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3edfe-a230-43b6-8444-e15450d525ec None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.402988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.411150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.411150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.411150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.411150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.411150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.552210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1209/2407] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:02:21 2022] POST /identity/v3/projects => generated 397 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:02:21.556606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.567575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb241e-b61e-482a-94a2-6caa1264702b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.625853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.626457 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.627253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.627253 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.627676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.630200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.765193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1199/2408] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:02:21 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:02:21.767211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.780387 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b92a8c-a604-484d-9f17-8e8219640c59 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.785358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.786001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.786462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.786773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.787535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.789066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.809802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1210/2409] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:02:21.810771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.828824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b003-5b29-465c-a678-32b3c08d46ab admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.839917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.843508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.843700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.844032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.844702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.849987 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccae860-821d-4481-b64b-d3b912acfc8d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.865690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1200/2410] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:02:21.873135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.884272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09957d5d-e848-4384-a5c1-f3f7dbc54d0b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.891423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.892159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.892414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.892690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] PATH_INFO: `/v3/projects/4e960456df854c7ab6235c12b2b43ab8/users/1924bd39cd6f41828fd08da4c5aac79a/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.908327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e960456df854c7ab6235c12b2b43ab8, user_id=1924bd39cd6f41828fd08da4c5aac79a, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.911590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:22.008567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1211/2411] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:02:21 2022] PUT /identity/v3/projects/4e960456df854c7ab6235c12b2b43ab8/users/1924bd39cd6f41828fd08da4c5aac79a/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:22.029567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b673-1ca4-47fd-8ffc-0ee5535e579c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:22.030161 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b673-1ca4-47fd-8ffc-0ee5535e579c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:22.030801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b673-1ca4-47fd-8ffc-0ee5535e579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:22.064762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-82a2b673-1ca4-47fd-8ffc-0ee5535e579c None None] MFA Rules not processed for user `1924bd39cd6f41828fd08da4c5aac79a`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:22.065989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-82a2b673-1ca4-47fd-8ffc-0ee5535e579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:22.154444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1201/2412] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:22.169277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:22.171560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:22.288801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52da46-6c37-4c20-981d-4c9e950561ab admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:22.294512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:22.295432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:22.295432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:22.295741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:22.310886 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:22.313604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:22.331668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1212/2413] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:23.574753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:23.575246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:23.575420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:23.600916 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] MFA Rules not processed for user `3f9025a5d28b47d8bc36f131e908382f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:23.602151 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:23.663458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1202/2414] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:02:24.481466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:24.483601 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:24.614604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cab0a-9234-4c55-843d-e07d43b1aae9 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:24.626934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:24.626934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:24.626934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:24.626934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:24.629816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:24.644449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:24.646547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:24.657644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1213/2415] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:24.662162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:24.664221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:24.677277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:24.739845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24eb88-7707-4ee4-bed6-96715522d983 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:24.745059 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:24.745820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:24.746102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:24.746377 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] PATH_INFO: `/v3/users/5171bf0514c3474aae358a9d7cfaf538` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:24.747609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5171bf0514c3474aae358a9d7cfaf538)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:24.749897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:25.069427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5171bf0514c3474aae358a9d7cfaf538'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.115156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5171bf0514c3474aae358a9d7cfaf538'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.122994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5171bf0514c3474aae358a9d7cfaf538'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.123700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5171bf0514c3474aae358a9d7cfaf538'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.135475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6d35cdc3-7cb5-46ad-b91f-e2ee5cbdd062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5171bf0514c3474aae358a9d7cfaf538'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.141269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1203/2416] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:02:24 2022] DELETE /identity/v3/users/5171bf0514c3474aae358a9d7cfaf538 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:25.408116 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:25.425328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119607f-9d16-4a65-911b-8be7835ea200 service placement] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:25.459194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:25.484405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:25.489889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:25.489889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:25.489889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] PATH_INFO: `/v3/projects/9ce953ed361742a29b0c750ca13d88e2` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:25.489889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce953ed361742a29b0c750ca13d88e2)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:25.489889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:25.681145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce953ed361742a29b0c750ca13d88e2', 'request_id': 'req-bcc84c68-0ee0-441b-8b55-7049404595cb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.686567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce953ed361742a29b0c750ca13d88e2', 'request_id': 'req-bcc84c68-0ee0-441b-8b55-7049404595cb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.777923 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce953ed361742a29b0c750ca13d88e2', 'request_id': 'req-bcc84c68-0ee0-441b-8b55-7049404595cb'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.790337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] The token cache is being invalidate because project 9ce953ed361742a29b0c750ca13d88e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:02:25.790633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.791336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bcc84c68-0ee0-441b-8b55-7049404595cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:25.793176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1214/2417] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:02:25 2022] DELETE /identity/v3/projects/9ce953ed361742a29b0c750ca13d88e2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:25.798391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:25.798921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:25.799131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:25.826538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:25.827712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:25.895013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1204/2418] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:02:25.903073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:25.903396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:25.903678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:25.924542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:25.925782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:25.933829 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1215/2419] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:25.938168 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:25.948570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56694-8d68-4a88-9219-59f713f3962f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:25.953055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:25.953659 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:25.953935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:25.954204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:25.954893 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:02:25.955194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:25.957471 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:25.968831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1205/2420] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:02:25.973741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:25.983282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067bc3-a01e-451c-b12e-b716a20c9832 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:25.988468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:25.989068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:25.989355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:25.989630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:25.990291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:25.992236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.094113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1216/2421] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:02:25 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:02:26.098307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:26.108685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766c93-4d52-4a8a-9258-fd39a8d086fa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:26.117926 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:26.164022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:26.164620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.164895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.165160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.165777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:26.168022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.264165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1206/2422] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:02:26 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:02:26.271705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:26.282606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6d23-ce1e-4827-b381-8b62b2685508 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:26.287338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:26.287939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.288283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.288525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.289195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:26.290971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.301656 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1217/2423] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:02:26.306263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:26.316967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003e7d1-df2a-4089-a28c-a9833d8a410b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:26.321576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:26.322172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.322450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.322751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.323394 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:26.324984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba2a89-e5ba-4993-b112-ba0173710350 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.336010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1207/2424] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:02:26.340918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:26.352113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe8ba6-aa5b-40de-8e7f-0e747286ce45 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:26.356547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:26.357193 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.357471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.357743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] PATH_INFO: `/v3/projects/4fab402e38f1412b8c6b895788d11a28/users/8bec7c9367ee4c00a104cbe3753ee9ce/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.373173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fab402e38f1412b8c6b895788d11a28, user_id=8bec7c9367ee4c00a104cbe3753ee9ce, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:26.376127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.452596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1218/2425] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:02:26 2022] PUT /identity/v3/projects/4fab402e38f1412b8c6b895788d11a28/users/8bec7c9367ee4c00a104cbe3753ee9ce/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:26.461857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.462145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.462423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.483249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:26.484463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:26.545074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1208/2426] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:26.555361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:26.557120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:26.616368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a0bb2-7a6b-4a12-b0fd-c184961ddc86 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:26.619971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:26.620581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:26.620861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:26.621131 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:26.630583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:26.632843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c961432-a5ba-4941-839f-d8b170b3d44d service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:26.641516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1219/2427] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:32.011993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:32.014352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:32.052592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325dc56-3cf8-442c-9754-a9e23410c327 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:32.064903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:32.067072 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:32.071599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:32.071599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] PATH_INFO: `/v3/users/1924bd39cd6f41828fd08da4c5aac79a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:32.071599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1924bd39cd6f41828fd08da4c5aac79a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:32.072271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:32.310485 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1924bd39cd6f41828fd08da4c5aac79a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:32.388744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1924bd39cd6f41828fd08da4c5aac79a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:32.406460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1924bd39cd6f41828fd08da4c5aac79a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:32.408094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1924bd39cd6f41828fd08da4c5aac79a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:32.437539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1924bd39cd6f41828fd08da4c5aac79a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:32.447956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1209/2428] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:02:32 2022] DELETE /identity/v3/users/1924bd39cd6f41828fd08da4c5aac79a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:32.453188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:32.454704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:32.455092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:32.466464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:32.480903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:32.482633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:32.580553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1220/2429] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:32.602641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:32.605850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:32.614601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:32.705423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d011840-4707-46eb-b8ba-369030664ea6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:32.718209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:32.719960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:32.720444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:32.720887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:32.738997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:32.741338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902c4a0-163c-48cb-9510-2328599c1b74 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:32.754568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1210/2430] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:33.392019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:33.393555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:33.406977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c59d11-dc9a-476e-aa22-7cd282db6481 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:33.413406 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:33.414738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:33.415445 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:33.415913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] PATH_INFO: `/v3/projects/4e960456df854c7ab6235c12b2b43ab8` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:33.417552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e960456df854c7ab6235c12b2b43ab8)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:33.420001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:33.580512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e960456df854c7ab6235c12b2b43ab8', 'request_id': 'req-ae067947-e181-4ab3-99fc-6599707b2b79'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:33.589264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e960456df854c7ab6235c12b2b43ab8', 'request_id': 'req-ae067947-e181-4ab3-99fc-6599707b2b79'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:33.693395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e960456df854c7ab6235c12b2b43ab8', 'request_id': 'req-ae067947-e181-4ab3-99fc-6599707b2b79'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:33.707953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] The token cache is being invalidate because project 4e960456df854c7ab6235c12b2b43ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:02:33.708250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:33.715701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:33.717612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1221/2431] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:02:33 2022] DELETE /identity/v3/projects/4e960456df854c7ab6235c12b2b43ab8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:34.525940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acfb1f-4fbe-4dd8-b52f-13b17823fe59 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:34.527037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acfb1f-4fbe-4dd8-b52f-13b17823fe59 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:34.527405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acfb1f-4fbe-4dd8-b52f-13b17823fe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:34.565398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-44acfb1f-4fbe-4dd8-b52f-13b17823fe59 None None] MFA Rules not processed for user `8bec7c9367ee4c00a104cbe3753ee9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:34.566757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-44acfb1f-4fbe-4dd8-b52f-13b17823fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:34.642766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1211/2432] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:34.659240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:34.660780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:34.742298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae067947-e181-4ab3-99fc-6599707b2b79 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:34.748599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:34.749320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:34.749674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:34.750015 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:34.759422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:34.761044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:34.771159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1222/2433] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:37.516202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:37.516821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:37.516821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:37.540566 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] MFA Rules not processed for user `8bec7c9367ee4c00a104cbe3753ee9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:37.541786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:37.562901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1212/2434] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:02:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:37.895336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:37.896972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:37.912683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:37.965416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213ed20-075e-4960-9599-f6508b5a0b90 service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:37.969779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:37.970412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:37.970725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:37.971984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:37.973785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:37.984895 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:37.986341 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663735-8b29-4507-9a5d-863cdd3acf97 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:37.995616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1223/2435] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:39.173023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:39.187415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3a5ae-1695-4802-a6bf-ce97a077de81 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:39.194004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:39.194616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:39.195011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:39.195189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:39.196922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:39.210237 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:39.211897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:39.224492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1213/2436] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:48.024488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=20 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:49.218942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=65 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:52.099841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f999f-b68f-4ff4-8f21-dca3e963c244 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:52.100800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f999f-b68f-4ff4-8f21-dca3e963c244 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:52.101159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f999f-b68f-4ff4-8f21-dca3e963c244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:52.121244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a71f999f-b68f-4ff4-8f21-dca3e963c244 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:52.122374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a71f999f-b68f-4ff4-8f21-dca3e963c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:52.187146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1224/2437] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:02:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:52.342539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:52.343994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:52.410787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7a5be-b36a-4ae4-b24d-bbb37a57ad97 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:52.415877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:52.416491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:52.416766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:52.417039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:52.426056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:52.427718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdda9c-2d46-4be1-b06c-b7c690afaabf service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:52.436487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1214/2438] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:02:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:03:02.178997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=25 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:02.432864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=32 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:09.163737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.168794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.168794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.168794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1225/2439] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:03:09.173738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f10-4239-461a-b086-58c2cf3e81b7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.174316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f10-4239-461a-b086-58c2cf3e81b7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.174621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f10-4239-461a-b086-58c2cf3e81b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.197441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4a890f10-4239-461a-b086-58c2cf3e81b7 None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:09.198910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a890f10-4239-461a-b086-58c2cf3e81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:09.210282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1215/2440] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:09.223127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:09.224852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:09.284605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa194c-bb56-4128-8a74-b609a8808b6d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:09.288166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:09.288709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.288991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.289301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.299437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:09.300910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51649b3-33d4-415b-8e35-57c487bf6412 service glance] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:09.312539 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1226/2441] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:09.516952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ae59-5971-4261-83b3-8f5ff13f42c8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.517241 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ae59-5971-4261-83b3-8f5ff13f42c8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.517523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ae59-5971-4261-83b3-8f5ff13f42c8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.518937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1216/2442] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:03:09.524030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e385-e7db-41e8-b6e8-9c95e4528ed0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.524391 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e385-e7db-41e8-b6e8-9c95e4528ed0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.524736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e385-e7db-41e8-b6e8-9c95e4528ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.544861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-13d7e385-e7db-41e8-b6e8-9c95e4528ed0 None None] MFA Rules not processed for user `06a32a4ecd71495785ec867dafc1c8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:09.545974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-13d7e385-e7db-41e8-b6e8-9c95e4528ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:09.554386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1227/2443] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:09.567370 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2ae59-5971-4261-83b3-8f5ff13f42c8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:09.578888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2ae59-5971-4261-83b3-8f5ff13f42c8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:09.582443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:09.583173 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.583454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.583728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.590942 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:09.592381 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2580f3d-a0e9-48bd-bba9-466425e08333 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:09.600276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1217/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:09.683299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.683787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.684166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] PATH_INFO: `/` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.685683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1228/2445] 10.222.0.64 () {54 vars in 921 bytes} [Wed Aug 24 09:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:03:09.691090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e2c4-9894-4545-ac4b-1e1d8773879d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.692103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e2c4-9894-4545-ac4b-1e1d8773879d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.692379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e2c4-9894-4545-ac4b-1e1d8773879d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.712265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f521e2c4-9894-4545-ac4b-1e1d8773879d None None] MFA Rules not processed for user `0880178d7ea842afba2a0a4b8bd6d842`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:09.713469 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f521e2c4-9894-4545-ac4b-1e1d8773879d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:09.722559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1218/2446] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Aug 24 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:09.733316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:09.735038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:09.748607 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38333aa-ae00-42ba-b4c1-3653a147e27f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:09.752181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:09.752774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:09.753054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:09.753342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:09.764491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:09.765926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:09.776561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1229/2447] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:12.151193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:12.151697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:12.151806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:12.172111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:12.173331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:12.181330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1219/2448] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:03:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:12.193844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:12.205052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c245e-1159-41b9-8f5e-2677f31a3fad service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:12.210809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:12.211414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:12.211690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:12.211960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:12.221149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:12.222766 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf93e2-654f-40da-b073-80a007c8003c service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:12.230672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1230/2449] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:03:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:03:22.166522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=15 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:22.227563 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=41 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:26.401017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:26.402563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:26.468007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137088c-1ee6-4a1d-8cd0-c76fe4e6ef3a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:26.471782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:26.472399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:26.472674 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:26.472944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:26.475286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:26.538263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:26.539762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:26.550263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1220/2450] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:26.883833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a99bee-941a-4899-85c6-eefb086b5e54 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:26.884316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a99bee-941a-4899-85c6-eefb086b5e54 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:26.884412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a99bee-941a-4899-85c6-eefb086b5e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:26.903969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-15a99bee-941a-4899-85c6-eefb086b5e54 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:26.905183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-15a99bee-941a-4899-85c6-eefb086b5e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:26.914159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1231/2451] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 09:03:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:26.927061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:26.938772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da03a7-58ee-4f7d-909a-f3c9ec6d01cf service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:26.943869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:26.944483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:26.944768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:26.945044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:26.964046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:26.966824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:26.977307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1221/2452] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:03:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:03:35.246602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:35.247227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:35.247494 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:35.269942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] MFA Rules not processed for user `f1381dae2b774b48a590e21b3d30dbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:35.271181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:35.291040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1232/2453] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:36.972389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=54 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:45.264431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:47.054018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.063673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411372d8-4e1c-44eb-8e01-35e2b5e2eab8 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.067243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.067871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.068174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.068419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.070787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:47.135894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.137269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.148389 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1222/2454] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:51.591919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:51.593379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:51.611678 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8085ed6-3f8e-463a-9a4f-6ba5209d54a8 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:51.616852 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:51.617220 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.617504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.617781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] PATH_INFO: `/v3/users/3f9025a5d28b47d8bc36f131e908382f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.619087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9025a5d28b47d8bc36f131e908382f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:51.621006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:51.821764 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9025a5d28b47d8bc36f131e908382f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.822524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9025a5d28b47d8bc36f131e908382f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.898709 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9025a5d28b47d8bc36f131e908382f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.906234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9025a5d28b47d8bc36f131e908382f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.920237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-90a2ba9d-4806-483a-ac6d-20916dd3c755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9025a5d28b47d8bc36f131e908382f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.926772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1233/2455] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:51 2022] DELETE /identity/v3/users/3f9025a5d28b47d8bc36f131e908382f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:52.254135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.266359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da6224-45e5-44bb-a529-2cda511652c2 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.334908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.335726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.336011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.336295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] PATH_INFO: `/v3/projects/574887b9d0fc42ce917c9b8b2c09e291` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.338528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574887b9d0fc42ce917c9b8b2c09e291)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.343805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.443698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574887b9d0fc42ce917c9b8b2c09e291', 'request_id': 'req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.449649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574887b9d0fc42ce917c9b8b2c09e291', 'request_id': 'req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.504158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574887b9d0fc42ce917c9b8b2c09e291', 'request_id': 'req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.516901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] The token cache is being invalidate because project 574887b9d0fc42ce917c9b8b2c09e291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:52.517172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.517844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-49cbfc27-2b62-43ad-958f-f9e67d4c5b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.519636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1223/2456] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:52 2022] DELETE /identity/v3/projects/574887b9d0fc42ce917c9b8b2c09e291 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:52.525315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.525745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.526049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.550726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:52.551898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.630453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1234/2457] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:52.639649 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.640132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.640584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.663023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:52.666812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.674165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1224/2458] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:52.678679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.689208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587311b-c4cc-4f7b-a5d4-8bd8b247ce26 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.695226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.695915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.696203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.696527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.697181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:52.697484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.699112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.711887 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1235/2459] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:03:52.717109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.733764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386338d-93a9-47f2-929a-7593362f3ab5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.741460 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.742549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.743095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.743592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.744752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.748570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.775404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:52.917039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1225/2460] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/projects => generated 403 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:52.921937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.934963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde4871-7a74-43d2-84a3-9fdd9a5a8067 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.992661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.993273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.993615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.993883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.994500 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.996427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.004351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:53.098645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1236/2461] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/users => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:53.103055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.113645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca57-8bc8-43f5-b446-37eaf7e9e218 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.119655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.119655 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.119899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.120155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.120810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.122424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.133848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1226/2462] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:53.138095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.148778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef96ace-5b0c-4434-b45b-71eced6e257b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.153929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.154469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.154786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.155047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.155687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.157315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421ca8-d6c5-4f5e-92e3-7f921263234a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.169661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1237/2463] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:53.174608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.186032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d0b6-81df-406c-868a-174cbaaa9707 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.190787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.191454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.191732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.192007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] PATH_INFO: `/v3/projects/d913b8bcdeb34817a6e3e772a6ff3e07/users/ca4568633e924676a264d3cd3a7e7831/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.208284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d913b8bcdeb34817a6e3e772a6ff3e07, user_id=ca4568633e924676a264d3cd3a7e7831, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.211060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.288282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1227/2464] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:53 2022] PUT /identity/v3/projects/d913b8bcdeb34817a6e3e772a6ff3e07/users/ca4568633e924676a264d3cd3a7e7831/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:53.293431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4366-19eb-4cce-a191-f57367d4ee58 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.294076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4366-19eb-4cce-a191-f57367d4ee58 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.294359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4366-19eb-4cce-a191-f57367d4ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.315442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-292a4366-19eb-4cce-a191-f57367d4ee58 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:53.316693 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-292a4366-19eb-4cce-a191-f57367d4ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:53.383005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1238/2465] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:53.393651 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.395791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:53.465246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12781e-2f74-495a-944a-bda675c6849c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.470027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.470932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.471391 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.471775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.484819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.490277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.501137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1228/2466] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:54.904460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.905018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.905018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.936355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] MFA Rules not processed for user `ca4568633e924676a264d3cd3a7e7831`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:54.937605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:55.024404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1239/2467] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:55.026368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.038774 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f99d37-8f93-4691-b935-b0afc9ce4b68 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.044744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.045642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.046131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.046575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.047500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.051429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.141181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1229/2468] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:55.147641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.160460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f538bc-5245-4822-a7f8-10fd8748ea9d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.246342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.247000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.247282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.247558 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.248229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.250281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.361914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:55.368419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1240/2469] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/users => generated 309 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:03:55.373734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.384851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6953da5-f574-4498-be9f-0b4361e59462 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.390576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.391417 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.391812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.392235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.393010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.394854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.408099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1230/2470] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:55.415863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.426444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d778c-11d9-4a69-ae26-00092d00e199 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.431290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.431932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.432209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.432477 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] PATH_INFO: `/v3/projects/aa092197cab94448a7beabbfdb9967f7/users/1fc355abc5d4418abae2514580b39472/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.449239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa092197cab94448a7beabbfdb9967f7, user_id=1fc355abc5d4418abae2514580b39472, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.451991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.597651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1241/2471] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:55 2022] PUT /identity/v3/projects/aa092197cab94448a7beabbfdb9967f7/users/1fc355abc5d4418abae2514580b39472/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:55.602624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.615315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15beba5-01ba-4a66-a114-65d296e7d357 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.687439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.688488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.688796 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.689121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.689878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.691749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e23e8-d71f-4f26-9576-724395ba908e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.703269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1231/2472] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:55.707855 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.731726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19bb5-704a-4d19-989a-88d1ed0c042d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.737635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.738439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.738865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.739241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] PATH_INFO: `/v3/projects/aa092197cab94448a7beabbfdb9967f7/users/1fc355abc5d4418abae2514580b39472/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.741412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa092197cab94448a7beabbfdb9967f7, user_id=1fc355abc5d4418abae2514580b39472, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.744309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.758202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1242/2473] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:55 2022] PUT /identity/v3/projects/aa092197cab94448a7beabbfdb9967f7/users/1fc355abc5d4418abae2514580b39472/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:03:57.440882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.441455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.441558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.473507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] MFA Rules not processed for user `1fc355abc5d4418abae2514580b39472`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:57.474814 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:57.491662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:57.626837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1232/2474] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:58.054967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.057342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.126121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51feb2e-5183-4ebe-b9ac-dec126f13d7f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.129837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.130506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.130816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.131093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.133618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.207186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.209538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.223672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1243/2475] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:58.229488 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.231071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.311311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9264431-0fc2-4e0e-8f98-520f38ad660a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.316464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.317078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.317365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.317636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.320051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.332136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.335503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.344152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1233/2476] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:59.652535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.750127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9d58a-090b-4d7c-982b-17eb24bfe189 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.755927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.757505 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.757788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.758071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] PATH_INFO: `/v3/users/ca4568633e924676a264d3cd3a7e7831` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.759267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca4568633e924676a264d3cd3a7e7831)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.761250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.774402 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:00.016320 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca4568633e924676a264d3cd3a7e7831'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.016981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca4568633e924676a264d3cd3a7e7831'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.085882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca4568633e924676a264d3cd3a7e7831'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.092686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca4568633e924676a264d3cd3a7e7831'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.107740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca4568633e924676a264d3cd3a7e7831'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.114242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1244/2477] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:59 2022] DELETE /identity/v3/users/ca4568633e924676a264d3cd3a7e7831 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:00.523949 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.553258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5342f19-3f6e-40e1-860f-4460d45b8951 service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.653808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.654994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.655588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.656190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] PATH_INFO: `/v3/projects/d913b8bcdeb34817a6e3e772a6ff3e07` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.657930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d913b8bcdeb34817a6e3e772a6ff3e07)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.660047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.786282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:00.791530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd913b8bcdeb34817a6e3e772a6ff3e07', 'request_id': 'req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.798936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd913b8bcdeb34817a6e3e772a6ff3e07', 'request_id': 'req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.910031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd913b8bcdeb34817a6e3e772a6ff3e07', 'request_id': 'req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.924140 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] The token cache is being invalidate because project d913b8bcdeb34817a6e3e772a6ff3e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:00.924422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.925077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:00.926859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1234/2478] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:00 2022] DELETE /identity/v3/projects/d913b8bcdeb34817a6e3e772a6ff3e07 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:00.932157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.934378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:00.948818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5993-b184-4b45-bab9-b458150cf1a6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:01.030865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:01.031541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:01.031963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:01.032250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] PATH_INFO: `/v3/users/1fc355abc5d4418abae2514580b39472` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:01.033415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc355abc5d4418abae2514580b39472)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:01.035397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:01.232952 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc355abc5d4418abae2514580b39472'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.235092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc355abc5d4418abae2514580b39472'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.300836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc355abc5d4418abae2514580b39472'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.309951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc355abc5d4418abae2514580b39472'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.331831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-2bf93041-195e-4b76-9356-bc312147e700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc355abc5d4418abae2514580b39472'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.340012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1245/2479] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:00 2022] DELETE /identity/v3/users/1fc355abc5d4418abae2514580b39472 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:01.647218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:01.649271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:01.680115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77624-b7e3-4a51-b0bb-4f3dd02a7a6c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:01.774694 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:01.775459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:01.775844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:01.776272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] PATH_INFO: `/v3/projects/aa092197cab94448a7beabbfdb9967f7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:01.777568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa092197cab94448a7beabbfdb9967f7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:01.780087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:01.942230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa092197cab94448a7beabbfdb9967f7', 'request_id': 'req-b2f7d320-6bc4-429a-b84b-46aa019ed797'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:01.950757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa092197cab94448a7beabbfdb9967f7', 'request_id': 'req-b2f7d320-6bc4-429a-b84b-46aa019ed797'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:02.024297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa092197cab94448a7beabbfdb9967f7', 'request_id': 'req-b2f7d320-6bc4-429a-b84b-46aa019ed797'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:02.056788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] The token cache is being invalidate because project aa092197cab94448a7beabbfdb9967f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:02.057223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:02.057947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b2f7d320-6bc4-429a-b84b-46aa019ed797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:02.062717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1235/2480] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:01 2022] DELETE /identity/v3/projects/aa092197cab94448a7beabbfdb9967f7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:02.066128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.067129 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.067622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.081673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:02.152649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:02.153980 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:02.304109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1246/2481] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:02.312087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.313169 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.313453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.336402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:02.337607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:02.346750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1236/2482] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:02.351937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.365806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de6a7e-4cd2-45f4-8851-a0732a80d11a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.381033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.382333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.382622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.382978 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.383648 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:02.383953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.385600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.399913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1247/2483] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:02.403979 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.413480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507c2dd-6a6f-455f-8c05-a213987ba8a0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.418475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.419131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.419420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.419693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.420593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.422621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.552245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1237/2484] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:02 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:02.556561 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.566746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1764e-2df0-40bd-9032-017e24376054 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.631259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.632456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.632856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.633188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.633820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.635840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.732541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1248/2485] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:02 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:02.736795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.747712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1e10-bbb6-4551-8ed6-a7a0ee2fdb1c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.752246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.752841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.753120 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.753392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.754060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.755689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.766390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1238/2486] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:02.770288 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.777211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:02.781301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36cc3c-3b0c-4a02-95b0-f854519780a6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.786035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.786832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.786950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.787202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.787906 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.789542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56428ff4-f0b3-4d9b-bea5-20f71fad227e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.801076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1249/2487] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:02.805538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:02.815815 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97824e1-fa7b-43e2-ae1d-75d09cdfde36 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:02.821778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:02.822519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.822859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.823109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] PATH_INFO: `/v3/projects/bd1650d3381a43169435a4653ed231f4/users/24004c41383e473ba922db832da6db4a/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.832557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:02.840627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1650d3381a43169435a4653ed231f4, user_id=24004c41383e473ba922db832da6db4a, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:02.843452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:02.937476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1239/2488] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:02 2022] PUT /identity/v3/projects/bd1650d3381a43169435a4653ed231f4/users/24004c41383e473ba922db832da6db4a/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:02.948936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:02.949217 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:02.949493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:02.987900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] MFA Rules not processed for user `24004c41383e473ba922db832da6db4a`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:02.989036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.072768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1250/2489] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:03.077823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.147187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b39c0-28a9-40c2-86de-a71c0e727111 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.151994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.152597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.152880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.153153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.153980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.155947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.271995 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1240/2490] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:03 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:03.276354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.287138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae309b08-d8bc-49cd-a5ff-27fc51c8a4a7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.348630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.349250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.354753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.354753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.354753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.354753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.470697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1251/2491] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:03 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:03.472569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.485397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ea75-1d21-4967-85ad-e100e968bf39 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.493293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.493293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.493293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.493293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.494382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.494382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.508690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1241/2492] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:03.513047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.524949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047abb84-8310-495c-81d8-2c7314f44a4b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.532234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.532933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.533294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.533593 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] PATH_INFO: `/v3/projects/08f8389ebc4444cfa309f35a47ae4003/users/23f267b88e3a4b4381b90d87575ae93d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.559410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f8389ebc4444cfa309f35a47ae4003, user_id=23f267b88e3a4b4381b90d87575ae93d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.563609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.682751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1252/2493] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:03 2022] PUT /identity/v3/projects/08f8389ebc4444cfa309f35a47ae4003/users/23f267b88e3a4b4381b90d87575ae93d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:03.686821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.697750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18c49-2f37-4465-9cba-2153431c2a8c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.757664 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.758623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.758623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.758850 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.759496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.761188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1533f-d4be-4cde-88c6-9254b624f5c5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.773424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1242/2494] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:03.777999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.788577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4316-5e12-406c-af8f-132f7a2ecfb1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.793455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.794273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.794554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.794862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] PATH_INFO: `/v3/projects/08f8389ebc4444cfa309f35a47ae4003/users/23f267b88e3a4b4381b90d87575ae93d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.797139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f8389ebc4444cfa309f35a47ae4003, user_id=23f267b88e3a4b4381b90d87575ae93d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.799821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.811893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1253/2495] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:03 2022] PUT /identity/v3/projects/08f8389ebc4444cfa309f35a47ae4003/users/23f267b88e3a4b4381b90d87575ae93d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:03.825622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.825907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.826191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.857828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] MFA Rules not processed for user `23f267b88e3a4b4381b90d87575ae93d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:03.859785 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.911320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:03.935281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1243/2496] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:03.947075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.948851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.995805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:04.031968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929df38d-7812-4926-81ff-50a3f8293168 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:04.035745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:04.036377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:04.036651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:04.036928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:04.047347 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:04.048859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabebf9-1155-4311-a09b-c81622c808c9 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:04.070880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1254/2497] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:07.392060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:07.459187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f4d74-0c55-4e34-8aad-5e62654575e6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:07.464463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:07.465276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:07.465601 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:07.465912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] PATH_INFO: `/v3/users/24004c41383e473ba922db832da6db4a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:07.467302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24004c41383e473ba922db832da6db4a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:07.469313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:07.673204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24004c41383e473ba922db832da6db4a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:07.767500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24004c41383e473ba922db832da6db4a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:07.775958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24004c41383e473ba922db832da6db4a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:07.776795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24004c41383e473ba922db832da6db4a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:07.789847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24004c41383e473ba922db832da6db4a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:07.796308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1244/2498] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:07 2022] DELETE /identity/v3/users/24004c41383e473ba922db832da6db4a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:07.800998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:07.801474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:07.801572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:07.821521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:07.822836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:07.886062 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1255/2499] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:07.896697 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:07.898669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:07.956475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28f06d-8fd4-4d79-aa2c-6d61b5f9a6a9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:07.960536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:07.961136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:07.961411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:07.961680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:07.971090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:07.972534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:07.982550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1245/2500] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:08.452173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:08.454547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:08.476922 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cc05f-80c9-4602-975b-d65efaeee946 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:08.485082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:08.486210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:08.486633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:08.487134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] PATH_INFO: `/v3/projects/bd1650d3381a43169435a4653ed231f4` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:08.491551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1650d3381a43169435a4653ed231f4)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:08.491551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:08.635926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1650d3381a43169435a4653ed231f4', 'request_id': 'req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:08.641442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1650d3381a43169435a4653ed231f4', 'request_id': 'req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:08.755201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1650d3381a43169435a4653ed231f4', 'request_id': 'req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:08.767760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] The token cache is being invalidate because project bd1650d3381a43169435a4653ed231f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:08.768058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:08.768736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:08.770219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1256/2501] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:08 2022] DELETE /identity/v3/projects/bd1650d3381a43169435a4653ed231f4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:08.774409 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:08.780669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:08.784640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687c92d-2c6c-41e4-bc65-6b0bd92cbed8 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:08.844158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:08.844798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:08.845081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:08.845458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] PATH_INFO: `/v3/users/23f267b88e3a4b4381b90d87575ae93d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:08.846591 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f267b88e3a4b4381b90d87575ae93d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:08.848645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:09.101317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f267b88e3a4b4381b90d87575ae93d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.195822 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f267b88e3a4b4381b90d87575ae93d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.205738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f267b88e3a4b4381b90d87575ae93d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.211530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f267b88e3a4b4381b90d87575ae93d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.229878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9863103d-fc4a-41a9-93b1-453c8105c40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f267b88e3a4b4381b90d87575ae93d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.236260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1246/2502] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:08 2022] DELETE /identity/v3/users/23f267b88e3a4b4381b90d87575ae93d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:09.689420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:09.691065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:09.705170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5c4a8-1ab3-4443-907b-01ca2ac2e228 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:09.765564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:09.779074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:09.779897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:09.780308 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:09.780694 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] PATH_INFO: `/v3/projects/08f8389ebc4444cfa309f35a47ae4003` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:09.782059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f8389ebc4444cfa309f35a47ae4003)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:09.784258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:09.942103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f8389ebc4444cfa309f35a47ae4003', 'request_id': 'req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:09.955430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f8389ebc4444cfa309f35a47ae4003', 'request_id': 'req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:10.024097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f8389ebc4444cfa309f35a47ae4003', 'request_id': 'req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:10.037048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] The token cache is being invalidate because project 08f8389ebc4444cfa309f35a47ae4003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:10.037346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:10.037999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ff85cb91-83bb-4c8d-8c3d-e986ea84f42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:10.039534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1257/2503] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:09 2022] DELETE /identity/v3/projects/08f8389ebc4444cfa309f35a47ae4003 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:10.044602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.045076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.045203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.065823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:10.067071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:10.127572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1247/2504] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:10.136579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.136879 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.137159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.157263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:10.158482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:10.166772 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1258/2505] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:10.171616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.181732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec7c8a-5184-4085-8ab1-a2faf3fbfc4e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.186404 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.187038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.187321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.187586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.188234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:10.188531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.190129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.202500 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1248/2506] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:10.206938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.216658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91552efc-b01b-446c-b7ca-6ef9419d9d34 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.221556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.222284 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.222571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.223037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.223679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.225594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.335386 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1259/2507] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:10.339660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.349902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a8b5-d557-460c-969c-5551684b6934 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.357810 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:10.407476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.408087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.408365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.408638 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.409266 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.411255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.528685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1249/2508] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:10.533033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.544909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20014e-3b2a-4d6e-a305-78d6d29254a9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.550031 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.550684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.550992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.551265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.551919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.554206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.565135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1260/2509] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:10.569462 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.580179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebe8e2-0dc7-4c5e-9d43-3caaa4168d38 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.585263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.585878 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.586439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.586439 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.587249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.588804 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227762d6-c3bc-47dd-99ba-af2ccab35a57 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.601095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1250/2510] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:10.606154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.617789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a4867-a28a-44a7-a3dd-8867d8e421e1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.622498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.626041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.626041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.626041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] PATH_INFO: `/v3/projects/e5fb36cacf2944e19dbe45c656959d33/users/0ec7ee483fb14696ba363ca03cab5661/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.639122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fb36cacf2944e19dbe45c656959d33, user_id=0ec7ee483fb14696ba363ca03cab5661, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.641814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:10.737796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1261/2511] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:10 2022] PUT /identity/v3/projects/e5fb36cacf2944e19dbe45c656959d33/users/0ec7ee483fb14696ba363ca03cab5661/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:10.750012 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.750304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.750578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.778141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] MFA Rules not processed for user `0ec7ee483fb14696ba363ca03cab5661`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:10.779330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:10.847648 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1251/2512] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:10.852763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:10.918024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18403b-159d-4b3e-81de-0f807e8a3a0d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:10.924210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:10.925043 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:10.925501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:10.925971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:10.927503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:10.929821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.013285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:11.020466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1262/2513] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:10 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:11.025081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.036098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89b56b-b386-4bf3-a68f-a9bb4dad9207 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.096106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.096852 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.097007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.097311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.097997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.100625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.197757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1252/2514] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:11 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:11.203880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.221534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17e6a-fad0-454d-b113-39a451944dda admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.227175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.227820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.228102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.228370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.229012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.230617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.241631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1263/2515] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:11.245827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.256817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9fb49-4a27-409f-b7fd-ad6200ef393a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.261669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.262325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.262607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.262914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] PATH_INFO: `/v3/projects/4fe3e545f6b44630a5f709432d61c6ab/users/790f7fabcdd04860ac022c42eba5b979/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.266107 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:11.280589 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe3e545f6b44630a5f709432d61c6ab, user_id=790f7fabcdd04860ac022c42eba5b979, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.284347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.317573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.320117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.401493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1253/2516] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:11 2022] PUT /identity/v3/projects/4fe3e545f6b44630a5f709432d61c6ab/users/790f7fabcdd04860ac022c42eba5b979/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:11.405459 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.415358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3bfd1-b440-45b0-a490-06d05546c798 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.482113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1892ce-fdc2-463d-8375-3ca5629ce0e1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.485882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.486506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.488644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.488777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.488893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.488992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.488992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.488992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.488992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.489333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.490455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.500467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.502071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be41d56-154c-456c-928e-3c155c708a62 service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.506836 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1254/2517] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:11.508171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.513652 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1264/2518] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:11.525141 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4444b-0c90-400b-aa5b-525df1707490 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.530994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.531891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.532317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.532662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] PATH_INFO: `/v3/projects/4fe3e545f6b44630a5f709432d61c6ab/users/790f7fabcdd04860ac022c42eba5b979/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.534991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe3e545f6b44630a5f709432d61c6ab, user_id=790f7fabcdd04860ac022c42eba5b979, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.537805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.550672 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1255/2519] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:11 2022] PUT /identity/v3/projects/4fe3e545f6b44630a5f709432d61c6ab/users/790f7fabcdd04860ac022c42eba5b979/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:11.561493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.561828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.562335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.595332 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] MFA Rules not processed for user `790f7fabcdd04860ac022c42eba5b979`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:11.596551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.651965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:11.689482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1265/2520] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:11.704879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.706943 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.775464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef06508-54aa-4c43-908a-fc176b553672 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.785366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.786001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.786281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.786542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.798240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.800051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9093f-a38c-4435-9248-01ab9cc351c3 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.814513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1256/2521] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:11.909195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.988029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c9b5-b1b8-4872-a479-4eaae8b242c0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.993890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.994614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.994994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.995335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] PATH_INFO: `/v3/users/0ec7ee483fb14696ba363ca03cab5661` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.996577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec7ee483fb14696ba363ca03cab5661)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.998565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:12.185302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec7ee483fb14696ba363ca03cab5661'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.186602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec7ee483fb14696ba363ca03cab5661'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.275756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec7ee483fb14696ba363ca03cab5661'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.284018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec7ee483fb14696ba363ca03cab5661'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.296734 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec7ee483fb14696ba363ca03cab5661'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.304038 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1266/2522] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:11 2022] DELETE /identity/v3/users/0ec7ee483fb14696ba363ca03cab5661 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:12.308667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:12.309131 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:12.309224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:12.328222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:12.329420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:12.407918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1257/2523] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:12.418882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:12.420306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:12.483715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acaaba-fe22-46e3-a427-c979828fe1bc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:12.487525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:12.488225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:12.488590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:12.488938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:12.499231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:12.500918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:12.512177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1267/2524] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:12.972829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:12.977321 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:13.002523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc9331-04b3-46ad-80e2-442d8b7dbd11 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:13.031449 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:13.032505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:13.032827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:13.033133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] PATH_INFO: `/v3/projects/e5fb36cacf2944e19dbe45c656959d33` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:13.036009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fb36cacf2944e19dbe45c656959d33)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:13.040001 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:13.048272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:13.181348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fb36cacf2944e19dbe45c656959d33', 'request_id': 'req-023faee7-4afa-4914-b4f2-9d715111d47e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.189245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fb36cacf2944e19dbe45c656959d33', 'request_id': 'req-023faee7-4afa-4914-b4f2-9d715111d47e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.264609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fb36cacf2944e19dbe45c656959d33', 'request_id': 'req-023faee7-4afa-4914-b4f2-9d715111d47e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.288647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] The token cache is being invalidate because project e5fb36cacf2944e19dbe45c656959d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:13.288957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.289627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.292435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1258/2525] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:12 2022] DELETE /identity/v3/projects/e5fb36cacf2944e19dbe45c656959d33 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:13.296101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:13.309528 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8cc0e-8e1e-4365-8ee2-f1a4d33741ae service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:13.313594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:13.393798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:13.394757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:13.396079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:13.396079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] PATH_INFO: `/v3/users/790f7fabcdd04860ac022c42eba5b979` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:13.397044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=790f7fabcdd04860ac022c42eba5b979)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:13.399026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:13.755078 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790f7fabcdd04860ac022c42eba5b979'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.756025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790f7fabcdd04860ac022c42eba5b979'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.840873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790f7fabcdd04860ac022c42eba5b979'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.852915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790f7fabcdd04860ac022c42eba5b979'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.870926 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d2a00472-b997-4bb1-9570-8747c34223c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790f7fabcdd04860ac022c42eba5b979'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.881366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1268/2526] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:13 2022] DELETE /identity/v3/users/790f7fabcdd04860ac022c42eba5b979 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:14.500031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:14.502639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.551703 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023faee7-4afa-4914-b4f2-9d715111d47e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.629364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.634795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.634795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.634795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] PATH_INFO: `/v3/projects/4fe3e545f6b44630a5f709432d61c6ab` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.634795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe3e545f6b44630a5f709432d61c6ab)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.646851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:14.792456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe3e545f6b44630a5f709432d61c6ab', 'request_id': 'req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.799602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe3e545f6b44630a5f709432d61c6ab', 'request_id': 'req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.910897 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe3e545f6b44630a5f709432d61c6ab', 'request_id': 'req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.932689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] The token cache is being invalidate because project 4fe3e545f6b44630a5f709432d61c6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:14.932989 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.933668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b8f78b86-9e5a-4be0-8b76-2abaeb02cd0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.935301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1259/2527] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:14 2022] DELETE /identity/v3/projects/4fe3e545f6b44630a5f709432d61c6ab => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:14.942541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.942541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.943343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.969496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:14.970897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.046567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1269/2528] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:15.054420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.056940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.056940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.076297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:15.077610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.092324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1260/2529] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:15.094187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.110170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b2dd-afaf-451a-9314-160d3365cfb6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.117055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.117718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.118001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.118436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.119169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:15.119472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.121194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.135774 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1270/2530] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:15.143274 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.160025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8612a2c-0243-4526-96ff-32dda2db9471 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.168752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.169912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.170430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.171210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.172272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.175447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.194877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:15.315263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1261/2531] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:15.319618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.333162 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cea5f-0c27-415e-8572-0403e06591ef admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.344820 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:15.401737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.402567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.402901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.403181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.403830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.406232 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.533099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1271/2532] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:15.544264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.559854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26470b6f-c99f-4047-98b2-41a297d898f7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.571668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.573447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.573447 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.573777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.575009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.578801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.617922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1262/2533] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:15.626007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.646141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e0756-835a-4130-9879-3d32f489d685 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.657691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.658639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.659238 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.660000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.660944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.665017 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac233924-3509-4135-8e5c-47cf9e594f88 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.680194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1272/2534] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:15.685905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.711398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e82f07-3487-4fc9-b65f-e09666f08a52 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.718833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.719733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.720121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.720506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] PATH_INFO: `/v3/projects/b95bb11fb94c47c3a6ecab5da3293235/users/555f3b094c21410e8a039a8adf68d2bb/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.753821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95bb11fb94c47c3a6ecab5da3293235, user_id=555f3b094c21410e8a039a8adf68d2bb, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.757798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.930368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1263/2535] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:15 2022] PUT /identity/v3/projects/b95bb11fb94c47c3a6ecab5da3293235/users/555f3b094c21410e8a039a8adf68d2bb/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:15.948415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.948415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.948415 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.978423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] MFA Rules not processed for user `555f3b094c21410e8a039a8adf68d2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:15.980213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:16.082796 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1273/2536] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:16.087415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.207068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c430a-c025-44d2-b5e0-b5af63bcc0f4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.207068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.207068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.207068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.207598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.207598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.207598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.403493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1264/2537] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:16 2022] POST /identity/v3/projects => generated 387 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:16.407569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.418794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74787f0e-9b61-4e46-a039-b820af07f4ee None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.481768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.482296 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.482582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.482891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.483529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.485497 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.610069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1274/2538] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:16 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:16.614743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.627010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f21405-8aac-49ae-9c89-7684eeedbacd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.632406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.634331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.634331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.634331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.634331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.636218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.649165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1265/2539] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:16.653937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.666161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8609-3bbe-4b19-93c6-1b6745df37d3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.671672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.672384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.672670 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.672950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] PATH_INFO: `/v3/projects/a87809d654a34631968ae3d834e0aa7b/users/8c3867b1e43a40ed81b48138920e72ad/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.683645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:16.691471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87809d654a34631968ae3d834e0aa7b, user_id=8c3867b1e43a40ed81b48138920e72ad, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.694465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.801761 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1275/2540] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:16 2022] PUT /identity/v3/projects/a87809d654a34631968ae3d834e0aa7b/users/8c3867b1e43a40ed81b48138920e72ad/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:16.806695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.817159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976e895-768c-4b3f-b286-351040849943 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.875303 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.875938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.876214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.876477 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.877115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.878769 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f7f31-485e-4bd4-9441-39aef06aa4e9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.889188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1266/2541] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:16.894286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.912196 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cdf76-01f0-470d-99c7-dc0ead9b13e3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.917746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.918425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.918742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.926941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] PATH_INFO: `/v3/projects/a87809d654a34631968ae3d834e0aa7b/users/8c3867b1e43a40ed81b48138920e72ad/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.926941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87809d654a34631968ae3d834e0aa7b, user_id=8c3867b1e43a40ed81b48138920e72ad, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.926941 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.934881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1276/2542] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:16 2022] PUT /identity/v3/projects/a87809d654a34631968ae3d834e0aa7b/users/8c3867b1e43a40ed81b48138920e72ad/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:16.946473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.946856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.947135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.978706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] MFA Rules not processed for user `8c3867b1e43a40ed81b48138920e72ad`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:16.980511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:17.000329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:17.066421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1267/2543] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:17.083092 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:17.085612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:17.177212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bbb2-a5f7-4bfc-8e38-07cfb8ee6c14 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:17.181242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:17.181945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:17.182303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:17.182752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:17.193140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:17.195683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501f29-cca8-4184-95b9-cb85185c5e68 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:17.214070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1277/2544] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:17.376932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:17.453543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4038-9a94-4bd4-80ed-6cf846553076 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:17.459232 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:17.460578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:17.460765 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:17.461040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] PATH_INFO: `/v3/users/555f3b094c21410e8a039a8adf68d2bb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:17.462338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] RBAC: Authorizing `identity:delete_user(user_id=555f3b094c21410e8a039a8adf68d2bb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:17.464301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:17.650090 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555f3b094c21410e8a039a8adf68d2bb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:17.709382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555f3b094c21410e8a039a8adf68d2bb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:17.717136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555f3b094c21410e8a039a8adf68d2bb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:17.717832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555f3b094c21410e8a039a8adf68d2bb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:17.730203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555f3b094c21410e8a039a8adf68d2bb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:17.736207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1268/2545] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:17 2022] DELETE /identity/v3/users/555f3b094c21410e8a039a8adf68d2bb => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:17.741331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:17.741671 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:17.741956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:17.761596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:17.762850 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:17.828242 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1278/2546] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:17.838069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:17.839685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:17.882142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:17.901236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324604-c33f-41eb-907f-d360e07a106e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:17.904904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:17.905519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:17.905801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:17.906075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:17.917047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:17.918570 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:17.929395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1269/2547] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:18.480679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:18.480679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:18.507018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65d91c-ef15-4e8e-8765-22780aa06c3b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:18.511849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:18.512503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:18.512783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:18.513055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] PATH_INFO: `/v3/projects/b95bb11fb94c47c3a6ecab5da3293235` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:18.514191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95bb11fb94c47c3a6ecab5da3293235)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:18.516985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:18.596992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:18.598461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:18.610218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf36c4-8611-4c25-89aa-cd0c53f7c81b service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:18.614023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:18.614615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:18.614928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:18.615196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:18.617029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:18.629143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:18.630584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:18.640717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1270/2548] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:18.675633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95bb11fb94c47c3a6ecab5da3293235', 'request_id': 'req-38399f68-ce53-42b4-b501-d41c23ec3d81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:18.681267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95bb11fb94c47c3a6ecab5da3293235', 'request_id': 'req-38399f68-ce53-42b4-b501-d41c23ec3d81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:18.829757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95bb11fb94c47c3a6ecab5da3293235', 'request_id': 'req-38399f68-ce53-42b4-b501-d41c23ec3d81'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:18.839543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:18.843845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] The token cache is being invalidate because project b95bb11fb94c47c3a6ecab5da3293235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:18.844144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:18.844911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:18.846471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1279/2549] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:18 2022] DELETE /identity/v3/projects/b95bb11fb94c47c3a6ecab5da3293235 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:18.850665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:18.861511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ced6b-1036-4f24-847b-a8e75e42b314 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:18.924138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:18.924781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:18.925066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:18.925382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] PATH_INFO: `/v3/users/8c3867b1e43a40ed81b48138920e72ad` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:18.926635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3867b1e43a40ed81b48138920e72ad)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:18.928644 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:19.115095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3867b1e43a40ed81b48138920e72ad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.190159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3867b1e43a40ed81b48138920e72ad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.199914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3867b1e43a40ed81b48138920e72ad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.200758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3867b1e43a40ed81b48138920e72ad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.216284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bed90fc-0b6b-4437-9749-05bb9be05db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3867b1e43a40ed81b48138920e72ad'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.223599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1271/2550] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:18 2022] DELETE /identity/v3/users/8c3867b1e43a40ed81b48138920e72ad => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:19.657643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:19.659289 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:19.675393 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38399f68-ce53-42b4-b501-d41c23ec3d81 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:19.770760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:19.771981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:19.772916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:19.773200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] PATH_INFO: `/v3/projects/a87809d654a34631968ae3d834e0aa7b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:19.775209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87809d654a34631968ae3d834e0aa7b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:19.779412 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:19.956934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87809d654a34631968ae3d834e0aa7b', 'request_id': 'req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:19.968542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87809d654a34631968ae3d834e0aa7b', 'request_id': 'req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:20.024180 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87809d654a34631968ae3d834e0aa7b', 'request_id': 'req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:20.037314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] The token cache is being invalidate because project a87809d654a34631968ae3d834e0aa7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:20.041493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:20.041493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c3bcc0b9-92d2-42a2-8f0d-7d3ced466650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:20.041493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1280/2551] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:19 2022] DELETE /identity/v3/projects/a87809d654a34631968ae3d834e0aa7b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:20.046282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.046904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.046904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.073155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:20.074501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:20.128281 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:20.152271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1272/2552] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:20.161352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.161643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.162102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.187398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:20.188665 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:20.197951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1281/2553] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:20.202542 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.216061 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8c0df-d6cc-4aea-abf0-9bc3e53504a8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.221386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.222160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.222351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.222634 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.223361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:20.223667 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.226083 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:20.243000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1273/2554] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:20.252476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.262868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe2f26-6fb8-43d3-834e-daa9f00e069e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.268080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.268712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.268994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.269258 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.269911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.272529 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:20.513265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1282/2555] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:20 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:20.518164 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.532431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7da2-281c-437b-8272-7b9eef31d584 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.618647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.619512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.619874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.620076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.620698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.622912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:20.781881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1274/2556] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:20 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:20.785999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.797426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea3a-3cc6-4c51-bee8-9cf9bdd8efd0 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.802405 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.803559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.803841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.804106 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.804765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.806420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:20.817691 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1283/2557] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:20.823775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.841554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c664c-03e7-4684-abe1-426ddf2294f8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.850421 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.851670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.852157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.852531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.853863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.856333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a435db-c104-4aea-8d6a-9caad5053bdc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:20.876119 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1275/2558] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:20.886235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:20.911448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd35d9-b2bd-435b-89c5-2774ef9e8878 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:20.920067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:20.922188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:20.923093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:20.923882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] PATH_INFO: `/v3/projects/835f087d347e4b3589480caeb294c60f/users/2845a693af114f25b9322240fb4e5a2d/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:20.951737 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835f087d347e4b3589480caeb294c60f, user_id=2845a693af114f25b9322240fb4e5a2d, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:20.955935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:21.032097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1284/2559] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:20 2022] PUT /identity/v3/projects/835f087d347e4b3589480caeb294c60f/users/2845a693af114f25b9322240fb4e5a2d/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:21.051205 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:21.051845 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:21.052075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:21.090735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] MFA Rules not processed for user `2845a693af114f25b9322240fb4e5a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:21.092145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:21.176534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1276/2560] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:21.188946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:21.190754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:21.224161 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:21.269943 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f73f-c506-44fa-9a1b-fb076d0a6fc5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:21.274023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:21.274687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:21.274987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:21.275263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:21.293835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:21.297465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351f21-eb22-4fdf-a843-d0d1127aa9d9 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:21.315536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1285/2561] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:21.340137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:21.386842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:21.426270 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2eaaac-62d5-4d20-b63f-1a5f1b2f45f7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:21.436228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:21.437202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:21.437709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:21.438126 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] PATH_INFO: `/v3/users/2845a693af114f25b9322240fb4e5a2d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:21.440077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2845a693af114f25b9322240fb4e5a2d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:21.446199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:21.673066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2845a693af114f25b9322240fb4e5a2d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:21.756511 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2845a693af114f25b9322240fb4e5a2d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:21.765994 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2845a693af114f25b9322240fb4e5a2d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:21.772636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2845a693af114f25b9322240fb4e5a2d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:21.784334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2845a693af114f25b9322240fb4e5a2d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:21.791002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1277/2562] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:21 2022] DELETE /identity/v3/users/2845a693af114f25b9322240fb4e5a2d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:21.796027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:21.796618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:21.796618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:21.818471 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:21.820755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:21.910835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1286/2563] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:21.935574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:21.940130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:22.093187 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9879c9-86e2-445e-a273-f0f8ea87e6af admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:22.100999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:22.101944 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:22.102225 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:22.102531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:22.114430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:22.116516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcdf77-ff65-4e0b-b2ab-a824ee819458 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:22.127855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1278/2564] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:22.707492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:22.709596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:22.729172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ddf16-9da9-4c5b-a526-69cfe5269ec5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:22.739831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:22.741937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:22.742635 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:22.743842 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] PATH_INFO: `/v3/projects/835f087d347e4b3589480caeb294c60f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:22.746304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835f087d347e4b3589480caeb294c60f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:22.749118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:22.928932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835f087d347e4b3589480caeb294c60f', 'request_id': 'req-9a0cbe27-05b4-4d7f-beb4-993519207f33'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:22.944472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835f087d347e4b3589480caeb294c60f', 'request_id': 'req-9a0cbe27-05b4-4d7f-beb4-993519207f33'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.047409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835f087d347e4b3589480caeb294c60f', 'request_id': 'req-9a0cbe27-05b4-4d7f-beb4-993519207f33'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.060962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] The token cache is being invalidate because project 835f087d347e4b3589480caeb294c60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:23.061285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.061927 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9a0cbe27-05b4-4d7f-beb4-993519207f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.063584 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1287/2565] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:22 2022] DELETE /identity/v3/projects/835f087d347e4b3589480caeb294c60f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:23.069965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.070470 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.071039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.097016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:23.098214 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:23.152306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:23.178928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1279/2566] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:23.186990 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.187556 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.187900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.212460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:23.213729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:23.228418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1288/2567] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:23.233594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.245833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25664407-4382-4a5d-b116-5435a67ebde1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.258758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.258758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.258758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.258758 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.259165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:23.259165 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.259580 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.286329 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1280/2568] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:23.290583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.301818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2e1a8-14dd-4b68-8753-1e0b842a8117 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.308192 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.309050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.309344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.310153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.310598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.312661 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.423531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1289/2569] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:23.428049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.438766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53219d5-f66a-445b-8114-70bfe0b095b7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.503489 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.504277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.504630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.504964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.505628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.509543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.519738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.522363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:23.539760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eadb1-c1aa-4d39-90e6-43a80b9127b7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.545517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.547823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.547823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.547823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] PATH_INFO: `/v3/users/2cd91f6976174fbf9692e0973435a4d5` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.548157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd91f6976174fbf9692e0973435a4d5)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.550202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.575942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:23.607786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1281/2570] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:23.613983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.625361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06ae7c-f566-435b-90ed-63cda12ab67f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.637307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.637307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.637307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.637307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.638098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.638098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.647743 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1282/2571] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:23.652220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.663110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae124-c12e-41f1-a532-a893ab3345b0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.668615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.669279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.669621 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.669951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.670776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.673122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.685330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1283/2572] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:23.689662 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.701503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41097302-be79-4e75-ad19-45a27c7469ec admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:23.706550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:23.707254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.707546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.707854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] PATH_INFO: `/v3/projects/999dc516fc634d62a83e97a99c9d68e9/users/e4dee4300d2a4a9ea3084c560da0de94/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.724685 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999dc516fc634d62a83e97a99c9d68e9, user_id=e4dee4300d2a4a9ea3084c560da0de94, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:23.727985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb395e6b-3d9b-4d45-b2b3-766206fb6827 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:23.764868 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd91f6976174fbf9692e0973435a4d5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.765696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd91f6976174fbf9692e0973435a4d5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.796801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1284/2573] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:23 2022] PUT /identity/v3/projects/999dc516fc634d62a83e97a99c9d68e9/users/e4dee4300d2a4a9ea3084c560da0de94/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:23.810130 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:23.812725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:23.812725 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:23.836186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd91f6976174fbf9692e0973435a4d5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.844834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd91f6976174fbf9692e0973435a4d5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.858250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] MFA Rules not processed for user `e4dee4300d2a4a9ea3084c560da0de94`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:23.860139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:23.870418 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd91f6976174fbf9692e0973435a4d5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:23.885519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1290/2574] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:23 2022] DELETE /identity/v3/users/2cd91f6976174fbf9692e0973435a4d5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:23.948315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1285/2575] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:23.953073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:23.955310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.040677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569062d-5bf6-4076-88db-a5678de2db4e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.049377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389910e6-b091-4783-b830-df214b389475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.050136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389910e6-b091-4783-b830-df214b389475 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.050421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389910e6-b091-4783-b830-df214b389475 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.050836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389910e6-b091-4783-b830-df214b389475 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.051550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389910e6-b091-4783-b830-df214b389475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.054102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389910e6-b091-4783-b830-df214b389475 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.203882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1291/2576] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:23 2022] POST /identity/v3/projects => generated 383 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:24.208424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.214306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:24.257555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389910e6-b091-4783-b830-df214b389475 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.260398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-389910e6-b091-4783-b830-df214b389475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.308276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f7ae-09a6-4919-b0dc-f1df8a70e9eb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.351080 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389910e6-b091-4783-b830-df214b389475 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.360101 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.361170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.361653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.362159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] PATH_INFO: `/v3/projects/dd27ac26660b46c388279e652200c7cd` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.364403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd27ac26660b46c388279e652200c7cd)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.368713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.397602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.398696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.399193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.399680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.400908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.404952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.416604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:24.534317 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1286/2577] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/users => generated 292 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:24.542997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.554877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6313faa-787d-4b8c-aebb-ac522f8d5c47 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.560434 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd27ac26660b46c388279e652200c7cd', 'request_id': 'req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:24.569940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd27ac26660b46c388279e652200c7cd', 'request_id': 'req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:24.623136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.623753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.624027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.624296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.624973 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.626673 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.637775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1287/2578] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:24.642368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.654798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59bc72-a37b-456c-b1d5-37a61e3df8e5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.659543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.660156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.660430 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.660700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.661349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.663615 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.675538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1288/2579] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:24.680221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.685240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd27ac26660b46c388279e652200c7cd', 'request_id': 'req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:24.692611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae883-3763-48b2-93ba-5722e0b548d7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.699011 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.699726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.700010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.700283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] PATH_INFO: `/v3/projects/dd4540458dab43ae8689da0db77ec0f2/users/40c8324831204656a3ad1b20340089b2/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.701121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] The token cache is being invalidate because project dd27ac26660b46c388279e652200c7cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:24.702166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:24.703174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed7b8e02-3cd4-42fe-902b-11f8f29d4e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:24.710799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1292/2580] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:24 2022] DELETE /identity/v3/projects/dd27ac26660b46c388279e652200c7cd => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:24.711278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e28b-3508-40b8-9fb4-1f1390db7d75 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.712255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e28b-3508-40b8-9fb4-1f1390db7d75 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.712255 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e28b-3508-40b8-9fb4-1f1390db7d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.718159 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4540458dab43ae8689da0db77ec0f2, user_id=40c8324831204656a3ad1b20340089b2, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.721023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f351737-cda8-4124-b9ed-e26cc729d855 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.736252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-5915e28b-3508-40b8-9fb4-1f1390db7d75 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:24.737679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-5915e28b-3508-40b8-9fb4-1f1390db7d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.746141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1293/2581] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:24.754641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.754984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.755214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.776704 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:24.777871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.786404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1294/2582] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:24.791856 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:24.797669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1289/2583] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:24 2022] PUT /identity/v3/projects/dd4540458dab43ae8689da0db77ec0f2/users/40c8324831204656a3ad1b20340089b2/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:24.799035 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.809891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f803c3-26b3-4a3a-9f92-9bea08d245a9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.868507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.868773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.869051 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.886424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.887120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.887403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.887673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.888317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:24.888640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.890269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.902340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] MFA Rules not processed for user `40c8324831204656a3ad1b20340089b2`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:24.907052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1295/2584] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:24.907230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.908553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.918791 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7b0e-7eae-448c-97ad-4446965c85c5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:24.923963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:24.925400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:24.925645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:24.925930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:24.926592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:24.928573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:24.979501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1290/2585] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:24.984067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:24.985386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:24.999066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61195e-4ec0-452d-9142-4de7467625f2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.004004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.004622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.004907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.005179 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.005851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.008481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.043785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1296/2586] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:25.048883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.114114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efa3-a5ea-46a0-95b4-369541dbd3b2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.118049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1291/2587] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:24 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:25.122406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.127823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.127823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.127823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.127823 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.128231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.128231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.188868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a472-9038-482b-a7f0-705ec0758e93 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.194178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.194843 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.195124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.195393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.196023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.198044 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.213216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:25.218279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1297/2588] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:25 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:25.223736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.236259 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da60f99-d29c-42e9-a08e-3e381d64d16f admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.241732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.242385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.242696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.242966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.243628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.245329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.255931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1298/2589] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:25.260564 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.270551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c2cb-05ff-482f-b740-6ea3a14ea4d1 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.275368 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.275988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.276269 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.276541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.277216 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.278860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.290194 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1299/2590] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:25.294597 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.305735 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77ebda-f276-4907-83ac-ede3bbb87000 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.312571 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.316088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1292/2591] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:25 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:25.316637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.316637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.316637 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] PATH_INFO: `/v3/projects/de6b3e9be33444d99f5908436c98e6ac/users/729b64c216894468924661175f03e765/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.317379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.329699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a222f7-9b54-4346-abb3-787aef56f5c1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.335128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.335759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.336160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.336645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.337139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.338818 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.347715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6b3e9be33444d99f5908436c98e6ac, user_id=729b64c216894468924661175f03e765, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.347715 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d2a8b-6178-40dd-b928-e3e31246e0dc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.355978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1293/2592] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:25.360375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.371952 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e537d-07ff-4fe9-b222-cceab08cde42 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.378375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.379087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.379397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.379677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] PATH_INFO: `/v3/projects/65797e299a934dfcb22f7b10a128d25a/users/0a96822cff1649f890ef03c44c4d9c72/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.396671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65797e299a934dfcb22f7b10a128d25a, user_id=0a96822cff1649f890ef03c44c4d9c72, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.399573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.454726 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1300/2593] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:25 2022] PUT /identity/v3/projects/de6b3e9be33444d99f5908436c98e6ac/users/729b64c216894468924661175f03e765/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:25.466981 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.467303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.467513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.500871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] MFA Rules not processed for user `729b64c216894468924661175f03e765`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:25.502109 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:25.534806 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1294/2594] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:25 2022] PUT /identity/v3/projects/65797e299a934dfcb22f7b10a128d25a/users/0a96822cff1649f890ef03c44c4d9c72/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:25.541283 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.581419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:25.587064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1301/2595] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:25.601915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.604139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:25.637729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b2f64-5326-45e2-a657-8bbc9e884d54 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.646574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.647777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.648243 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.648729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.649890 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.653221 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.673503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1295/2596] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:25.677970 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.689524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b210-9ec7-4575-8ce7-b6812f16b05b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.694736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.695437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.695717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.695996 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] PATH_INFO: `/v3/projects/65797e299a934dfcb22f7b10a128d25a/users/0a96822cff1649f890ef03c44c4d9c72/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.698352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=65797e299a934dfcb22f7b10a128d25a, user_id=0a96822cff1649f890ef03c44c4d9c72, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.701207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4acc2-609b-4bff-8461-e390addc38dd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.713279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1296/2597] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:25 2022] PUT /identity/v3/projects/65797e299a934dfcb22f7b10a128d25a/users/0a96822cff1649f890ef03c44c4d9c72/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:25.724380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.725238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.725425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.754778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30ee59-4746-4b9e-9b66-bbd25e98f1ed None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.761352 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] MFA Rules not processed for user `0a96822cff1649f890ef03c44c4d9c72`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:25.762600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:25.771655 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:25.861409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.862466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.863003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.863501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.872031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1297/2598] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:25.888554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:25.890431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:25.958553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a298fc9-378e-4c02-b9ef-782188581080 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:25.963197 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:25.963828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:25.964113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:25.964388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:25.966562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:25.973267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:25.976569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:25.989672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1302/2599] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:26.030597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:26.049552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:26.051380 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:26.063325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1298/2600] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:28.415518 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:28.416997 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:28.453444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ea39-168f-4560-9aac-76111a50e248 service nova] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:28.457900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:28.458527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:28.458942 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:28.459126 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:28.527037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:28.528513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187e3ff-7e22-4673-87b2-47c7739b37e9 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:28.539555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1303/2601] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:29.684413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:29.684413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a214e-295e-4dfb-841a-6f28e3a77d84 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:29.684413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:29.684413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.711008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.711008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] PATH_INFO: `/v3/users/729b64c216894468924661175f03e765` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.711008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=729b64c216894468924661175f03e765)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:29.711008 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:29.965390 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729b64c216894468924661175f03e765'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:30.065117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729b64c216894468924661175f03e765'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:30.078998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729b64c216894468924661175f03e765'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:30.080136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729b64c216894468924661175f03e765'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:30.107950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729b64c216894468924661175f03e765'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:30.108392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1299/2602] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:29 2022] DELETE /identity/v3/users/729b64c216894468924661175f03e765 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:30.116181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.116925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.116925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.137566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:30.138832 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.167380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:30.205006 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1304/2603] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:30.216037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.217808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.292420 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4317e7-aa92-4a87-a520-a30e0a474de8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.296493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.297252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.297532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.297808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.308886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.310366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.321531 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1300/2604] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:30.535686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.537846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.615706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a933b-1d05-4143-9956-9e673518046f None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.628763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.629863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.630369 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.631606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.635740 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.726535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.728042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328be433-d55f-49dd-bcd4-1e352df31cb4 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.738321 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1305/2605] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:30.834018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.836437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.851708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e1b-538e-4a35-834d-d13b52d701df service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.856873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.857562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.857842 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.858106 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] PATH_INFO: `/v3/projects/de6b3e9be33444d99f5908436c98e6ac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.859646 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6b3e9be33444d99f5908436c98e6ac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.861868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.864150 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:31.004523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6b3e9be33444d99f5908436c98e6ac', 'request_id': 'req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:31.012263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6b3e9be33444d99f5908436c98e6ac', 'request_id': 'req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:31.098888 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6b3e9be33444d99f5908436c98e6ac', 'request_id': 'req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:31.111857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] The token cache is being invalidate because project de6b3e9be33444d99f5908436c98e6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:31.112157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:31.112798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-29dfa6d1-a5ec-4df0-85f4-5b163a63888b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:31.114415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1301/2606] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:30 2022] DELETE /identity/v3/projects/de6b3e9be33444d99f5908436c98e6ac => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:31.128474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.129059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.129059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.150302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:31.151643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:31.229294 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1306/2607] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:31.237645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.237846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.238121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.259902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:31.261133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:31.269154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1302/2608] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:31.273552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.283485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b34f-f39f-45aa-961d-3c2acb8cb5e7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.288207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.291008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.291008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.291008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.291008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:31.291008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.291931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.304838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1307/2609] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:31.309532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.319562 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b29812-5a9b-48b7-943d-34ac3a99b2e1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.324316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.324925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.325203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.325479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.326219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.328194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.450201 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1303/2610] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:31.457270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.476305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8884c8-3737-4d2f-9ba1-d166c0935782 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.484334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:31.551182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.551812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.552093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.552362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.553680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.555745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.716846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1308/2611] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:31.722450 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.734052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61152c-61b0-44be-a08e-ae58a5ad3ffe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.738962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.741647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.741647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.741647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.741647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.743359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.754399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1304/2612] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:31.758591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.771683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d945bf-2a03-4d5e-a404-f62d5fa12360 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.776446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.777064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.777328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.777621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.778251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.779935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4006-5b8c-4a7d-912a-247626216175 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.790937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1309/2613] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:31.795653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.808287 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35923-1aaa-4aa2-8c8e-9bebab500839 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.813446 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.814174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.814461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.814777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] PATH_INFO: `/v3/projects/a3153673bf9c442ea4ab34da7ff88dbc/users/161c344ddcad4edb824ef4b0329f485f/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.830907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3153673bf9c442ea4ab34da7ff88dbc, user_id=161c344ddcad4edb824ef4b0329f485f, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.833669 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.932623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1305/2614] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:31 2022] PUT /identity/v3/projects/a3153673bf9c442ea4ab34da7ff88dbc/users/161c344ddcad4edb824ef4b0329f485f/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:31.943473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.943682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.943989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.100372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] MFA Rules not processed for user `161c344ddcad4edb824ef4b0329f485f`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:32.101621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:32.170336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1310/2615] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:32.175239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.242408 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368272e-dbea-4e94-90bb-a505cfd8559e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.247136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.248049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.248049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.248349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.248901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.250858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.426496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1306/2616] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:32 2022] POST /identity/v3/projects => generated 381 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:32.430993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.441847 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d3dc6-f48c-49b6-9dce-9a1a858ff84d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.507641 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.508250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.508525 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.508838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.509598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.511535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.613384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1311/2617] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:32 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:32.618092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.629142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c87ab-d057-4760-8974-25bd94dfba39 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.634536 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.635207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.635484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.635754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.636423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.638115 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.643778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:32.649588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1307/2618] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:32.653931 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.663958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b4cd-a075-4513-998b-2e6cca535d0e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.668457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.669135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.669414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.669687 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] PATH_INFO: `/v3/projects/94f4a54392b24ec29a51e281103ff5be/users/6545f1aacd7d4b1da29094e6df2669b6/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.672142 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:32.686290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f4a54392b24ec29a51e281103ff5be, user_id=6545f1aacd7d4b1da29094e6df2669b6, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.688968 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.835631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1312/2619] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:32 2022] PUT /identity/v3/projects/94f4a54392b24ec29a51e281103ff5be/users/6545f1aacd7d4b1da29094e6df2669b6/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:32.839670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.850196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaea27-8dd2-4b05-9433-d9e0a59f3e4e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.910444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.911057 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.911334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.911602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.912297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.913936 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bbeb6-ce77-4451-946f-370fbdbe7032 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.924519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1308/2620] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:32.929392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:32.939877 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a6fa-8991-4e0a-9002-306a8a5b3ba3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:32.945316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:32.945991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.946273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.946550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] PATH_INFO: `/v3/projects/94f4a54392b24ec29a51e281103ff5be/users/6545f1aacd7d4b1da29094e6df2669b6/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:32.949143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f4a54392b24ec29a51e281103ff5be, user_id=6545f1aacd7d4b1da29094e6df2669b6, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:32.952105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:32.962544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1313/2621] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:32 2022] PUT /identity/v3/projects/94f4a54392b24ec29a51e281103ff5be/users/6545f1aacd7d4b1da29094e6df2669b6/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:32.978275 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:32.979048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:32.979048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:33.008210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] MFA Rules not processed for user `6545f1aacd7d4b1da29094e6df2669b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:33.009476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:33.084244 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1309/2622] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:33.097795 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:33.099679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:33.158421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d253d3d-63ac-4253-a835-605ce3f0dd0d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:33.162151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:33.163290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:33.163575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:33.163844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:33.173001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:33.174465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:33.184526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1314/2623] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:34.544058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:34.554915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbd233-66a8-4af6-bdc3-cf55c06973ac None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:34.559754 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6545f1aacd7d4b1da29094e6df2669b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f4a54392b24ec29a51e281103ff5be', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:34.561812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:34.561812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:34.561812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:34.561812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:34.561812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:34.563508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:34.576624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1310/2624] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:34.580342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:34.590484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed1c4-d9ee-438d-aa1e-0dde2d2c4ae8 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:34.596181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6545f1aacd7d4b1da29094e6df2669b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f4a54392b24ec29a51e281103ff5be', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:34.596816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:34.597103 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:34.597381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:34.598086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:34.600910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:34.776698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1315/2625] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:34 2022] POST /identity/v3/projects => generated 352 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:34.781496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:34.792775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b20ed0-32f8-490a-ba96-08b29a3a5213 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:34.854031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6545f1aacd7d4b1da29094e6df2669b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f4a54392b24ec29a51e281103ff5be', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:34.854565 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:34.854874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:34.855149 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:34.855808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:34.856248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:34.857985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:34.877526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1311/2626] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:34.919492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:34.930170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c0c0-25af-4b7d-b488-401d5f18ba74 tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:34.934841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6545f1aacd7d4b1da29094e6df2669b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f4a54392b24ec29a51e281103ff5be', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:34.935478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:34.935765 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:34.936117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] PATH_INFO: `/v3/projects/7e02cafdf2d5407f9bcbb59fa332036b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:34.937277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7e02cafdf2d5407f9bcbb59fa332036b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:34.939213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:35.050903 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e02cafdf2d5407f9bcbb59fa332036b', 'request_id': 'req-4848beae-286b-44d0-85d4-a63f0350545b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:35.056936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e02cafdf2d5407f9bcbb59fa332036b', 'request_id': 'req-4848beae-286b-44d0-85d4-a63f0350545b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:35.132970 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e02cafdf2d5407f9bcbb59fa332036b', 'request_id': 'req-4848beae-286b-44d0-85d4-a63f0350545b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:35.145318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] The token cache is being invalidate because project 7e02cafdf2d5407f9bcbb59fa332036b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:35.145602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:35.146299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:35.148283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1316/2627] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:34 2022] DELETE /identity/v3/projects/7e02cafdf2d5407f9bcbb59fa332036b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:35.776577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:35.778369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:35.783950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:35.851084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbe491-6cfc-4037-a951-9bc6e60782de tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:35.892052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:35.893573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:35.923019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:35.923693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:35.924036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:35.924312 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] PATH_INFO: `/v3/users/e4dee4300d2a4a9ea3084c560da0de94` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:35.925600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4dee4300d2a4a9ea3084c560da0de94)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:35.927581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:35.944424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:35.978534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848beae-286b-44d0-85d4-a63f0350545b tempest-VolumeQuotaClassesTest-1191054087 tempest-VolumeQuotaClassesTest-1191054087-project-admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:35.983984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:35.984817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:35.985620 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:35.986086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] PATH_INFO: `/v3/users/161c344ddcad4edb824ef4b0329f485f` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:35.987380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] RBAC: Authorizing `identity:delete_user(user_id=161c344ddcad4edb824ef4b0329f485f)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:35.990099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:36.195496 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4dee4300d2a4a9ea3084c560da0de94'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.205072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '161c344ddcad4edb824ef4b0329f485f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.208910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '161c344ddcad4edb824ef4b0329f485f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.304737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4dee4300d2a4a9ea3084c560da0de94'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.312599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4dee4300d2a4a9ea3084c560da0de94'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.313338 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4dee4300d2a4a9ea3084c560da0de94'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.326181 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b5a3ff03-7eab-4080-9cb7-56e065114944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4dee4300d2a4a9ea3084c560da0de94'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.332434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1312/2628] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:35 2022] DELETE /identity/v3/users/e4dee4300d2a4a9ea3084c560da0de94 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:36.337272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:36.337571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:36.337860 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:36.357538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:36.359158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:36.382277 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:36.400508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '161c344ddcad4edb824ef4b0329f485f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.407875 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '161c344ddcad4edb824ef4b0329f485f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.418318 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1313/2629] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:36.420590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d90bdcde-bf06-4436-83fb-871771af871f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '161c344ddcad4edb824ef4b0329f485f'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:36.427329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1317/2630] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:35 2022] DELETE /identity/v3/users/161c344ddcad4edb824ef4b0329f485f => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:36.428248 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:36.429647 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:36.431311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:36.431598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:36.431871 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:36.449995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:36.451150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:36.496991 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ab13b-eeb4-45e3-9070-1839650fd891 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:36.528520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1318/2631] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:36.542909 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:36.582017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:36.582640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:36.583077 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:36.583419 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:36.593905 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:36.595407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:36.606111 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1314/2632] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:36.609360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c011ca-f6da-42b9-b840-40c237be202b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:36.613649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:36.614437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:36.614437 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:36.614666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:36.622382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:36.624916 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:36.626389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:36.636839 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1319/2633] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:37.181532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:37.183117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:37.197616 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880384b-5238-4576-aef5-1c6d44418a2f service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:37.202945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:37.203863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:37.204213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:37.204504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] PATH_INFO: `/v3/projects/999dc516fc634d62a83e97a99c9d68e9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:37.205849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999dc516fc634d62a83e97a99c9d68e9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:37.216267 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:37.217861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:37.220018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:37.236608 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b389b1-7755-41cd-aecd-b9186b9e2370 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:37.242305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:37.243138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:37.243488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:37.243866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] PATH_INFO: `/v3/projects/a3153673bf9c442ea4ab34da7ff88dbc` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:37.245118 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3153673bf9c442ea4ab34da7ff88dbc)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:37.247351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:37.364068 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999dc516fc634d62a83e97a99c9d68e9', 'request_id': 'req-0268757f-60be-4507-aff4-70f172102e28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.370640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999dc516fc634d62a83e97a99c9d68e9', 'request_id': 'req-0268757f-60be-4507-aff4-70f172102e28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.398073 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3153673bf9c442ea4ab34da7ff88dbc', 'request_id': 'req-a807d363-7939-4d9f-9cd1-46b0b917c0c5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.403838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3153673bf9c442ea4ab34da7ff88dbc', 'request_id': 'req-a807d363-7939-4d9f-9cd1-46b0b917c0c5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.447238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999dc516fc634d62a83e97a99c9d68e9', 'request_id': 'req-0268757f-60be-4507-aff4-70f172102e28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.459148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] The token cache is being invalidate because project 999dc516fc634d62a83e97a99c9d68e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:37.459454 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.460118 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.461607 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1315/2634] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:37 2022] DELETE /identity/v3/projects/999dc516fc634d62a83e97a99c9d68e9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:37.465602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:37.466974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:37.481297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:37.524724 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3153673bf9c442ea4ab34da7ff88dbc', 'request_id': 'req-a807d363-7939-4d9f-9cd1-46b0b917c0c5'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.533372 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268757f-60be-4507-aff4-70f172102e28 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:37.539690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] The token cache is being invalidate because project a3153673bf9c442ea4ab34da7ff88dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:37.539690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.540243 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.542168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1320/2635] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:37 2022] DELETE /identity/v3/projects/a3153673bf9c442ea4ab34da7ff88dbc => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:37.545712 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:37.547521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:37.597194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:37.597838 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:37.598124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:37.598392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] PATH_INFO: `/v3/users/40c8324831204656a3ad1b20340089b2` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:37.599631 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c8324831204656a3ad1b20340089b2)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:37.602087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:37.614566 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807d363-7939-4d9f-9cd1-46b0b917c0c5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:37.620075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:37.620717 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:37.620996 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:37.621267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] PATH_INFO: `/v3/users/6545f1aacd7d4b1da29094e6df2669b6` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:37.622478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6545f1aacd7d4b1da29094e6df2669b6)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:37.624417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:37.798993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6545f1aacd7d4b1da29094e6df2669b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.799572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6545f1aacd7d4b1da29094e6df2669b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.800953 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c8324831204656a3ad1b20340089b2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.904102 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6545f1aacd7d4b1da29094e6df2669b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.911786 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6545f1aacd7d4b1da29094e6df2669b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.925204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6545f1aacd7d4b1da29094e6df2669b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.926770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:37.931815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1321/2636] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:37 2022] DELETE /identity/v3/users/6545f1aacd7d4b1da29094e6df2669b6 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:37.967135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c8324831204656a3ad1b20340089b2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.976162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c8324831204656a3ad1b20340089b2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.976906 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c8324831204656a3ad1b20340089b2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.997013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c8324831204656a3ad1b20340089b2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:37.997781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1316/2637] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:37 2022] DELETE /identity/v3/users/40c8324831204656a3ad1b20340089b2 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:38.489027 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:38.490680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:38.563105 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:38.564700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:38.585768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61a212-cf66-447c-b48b-932716f0e7c5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:38.612279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:38.636719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2e12-d4c5-4647-9ade-e10cdc3bb7aa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:38.641881 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:38.642538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:38.642853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:38.643136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] PATH_INFO: `/v3/projects/dd4540458dab43ae8689da0db77ec0f2` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:38.644299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4540458dab43ae8689da0db77ec0f2)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:38.646310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:38.654266 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:38.655083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:38.655358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:38.655653 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] PATH_INFO: `/v3/projects/94f4a54392b24ec29a51e281103ff5be` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:38.656825 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f4a54392b24ec29a51e281103ff5be)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:38.658979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:38.784735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4540458dab43ae8689da0db77ec0f2', 'request_id': 'req-4688cb28-82f2-429b-9237-44637ce5b140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.790230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4540458dab43ae8689da0db77ec0f2', 'request_id': 'req-4688cb28-82f2-429b-9237-44637ce5b140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.830408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f4a54392b24ec29a51e281103ff5be', 'request_id': 'req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.835833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f4a54392b24ec29a51e281103ff5be', 'request_id': 'req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.950099 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4540458dab43ae8689da0db77ec0f2', 'request_id': 'req-4688cb28-82f2-429b-9237-44637ce5b140'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.961805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] The token cache is being invalidate because project dd4540458dab43ae8689da0db77ec0f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:38.967467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.967467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:38.967467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1317/2638] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:38 2022] DELETE /identity/v3/projects/dd4540458dab43ae8689da0db77ec0f2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:38.968114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:38.969418 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.034040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688cb28-82f2-429b-9237-44637ce5b140 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.041636 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f4a54392b24ec29a51e281103ff5be', 'request_id': 'req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.055112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] The token cache is being invalidate because project 94f4a54392b24ec29a51e281103ff5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:39.060739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.060739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-63c6e8f3-02d8-48f9-9c67-2d883c1af4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.060739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1322/2639] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:38 2022] DELETE /identity/v3/projects/94f4a54392b24ec29a51e281103ff5be => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:39.062513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f575-ef20-40f4-bc42-e551461c524a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.062841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f575-ef20-40f4-bc42-e551461c524a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.063122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f575-ef20-40f4-bc42-e551461c524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.083545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f614f575-ef20-40f4-bc42-e551461c524a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:39.084836 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f614f575-ef20-40f4-bc42-e551461c524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.096847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.097492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.097771 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.098052 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] PATH_INFO: `/v3/users/0a96822cff1649f890ef03c44c4d9c72` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.099215 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a96822cff1649f890ef03c44c4d9c72)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.101665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.117158 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:39.120553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:39.145030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1323/2640] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:39.152692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.152979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.153250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.173770 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:39.176992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.183554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1324/2641] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:39.187948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.196985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e1fd-da31-4135-924a-a8f0099ca1a0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.205683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.205683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.205683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.205683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.206133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:39.206133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.206133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.218685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1325/2642] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:39.223247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.233394 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421b1e8-9d9f-436b-afb8-f5528c1d75e7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.238124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.238743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.239030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.239310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.239960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.242501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.290308 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a96822cff1649f890ef03c44c4d9c72'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.367604 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a96822cff1649f890ef03c44c4d9c72'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.368783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1326/2643] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:39 2022] POST /identity/v3/projects => generated 393 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:39.373188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.379324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a96822cff1649f890ef03c44c4d9c72'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.380028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a96822cff1649f890ef03c44c4d9c72'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.392378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a96822cff1649f890ef03c44c4d9c72'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:39.399781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1318/2644] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:38 2022] DELETE /identity/v3/users/0a96822cff1649f890ef03c44c4d9c72 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:39.443508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13da42-d949-4a5d-8258-58f10ec498da admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.507458 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.508124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.508342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.508611 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.509234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.511278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.648821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1327/2645] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:39 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:39.653691 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.655575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.670486 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4949-77d3-43f8-a78a-d01872d1f2ff admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.676424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.677067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.677463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.677622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.678292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.680055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.692456 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1319/2646] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:39.696683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.708112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bb20a-9351-42d6-b853-03343de11af6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.715565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.716263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.717059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.717059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.717773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.719818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2404ab1-d17d-4b48-8391-6bc1b5622186 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.739495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1328/2647] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:39.743924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.759368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf6820-2e5c-4b43-b37c-89f063aabb31 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:39.766982 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:39.767682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.767964 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.768242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] PATH_INFO: `/v3/projects/455d7682d283471e9f53e62b610c036f/users/4185c6d71815472d96cb9ecd044073e0/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.785596 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455d7682d283471e9f53e62b610c036f, user_id=4185c6d71815472d96cb9ecd044073e0, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:39.788516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:39.928963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1320/2648] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:39 2022] PUT /identity/v3/projects/455d7682d283471e9f53e62b610c036f/users/4185c6d71815472d96cb9ecd044073e0/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:39.939828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:39.940112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:39.940496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:39.957029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:39.963587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.974626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] MFA Rules not processed for user `4185c6d71815472d96cb9ecd044073e0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:39.975910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:39.988306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:40.041127 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d421b-6d70-4db8-9365-d94bf494a478 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.046175 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.051443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.051443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.051443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] PATH_INFO: `/v3/projects/65797e299a934dfcb22f7b10a128d25a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.051443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65797e299a934dfcb22f7b10a128d25a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.051443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.051897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1329/2649] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:40.055915 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.067072 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be5614-6826-49b0-aa19-1d70d69b67c7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.072899 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.073524 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.074251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.074251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.074818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.077133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.161903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65797e299a934dfcb22f7b10a128d25a', 'request_id': 'req-f1175a67-7e6d-47cf-a2a1-fadf4081678d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:40.167426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65797e299a934dfcb22f7b10a128d25a', 'request_id': 'req-f1175a67-7e6d-47cf-a2a1-fadf4081678d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:40.169993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:40.186372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1330/2650] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/projects => generated 393 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:40.190673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.235799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65797e299a934dfcb22f7b10a128d25a', 'request_id': 'req-f1175a67-7e6d-47cf-a2a1-fadf4081678d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:40.248993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] The token cache is being invalidate because project 65797e299a934dfcb22f7b10a128d25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:40.249301 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:40.249978 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f1175a67-7e6d-47cf-a2a1-fadf4081678d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:40.251575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1321/2651] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:39 2022] DELETE /identity/v3/projects/65797e299a934dfcb22f7b10a128d25a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:40.258166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139e93-25cf-4e62-95bf-1a3179725702 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.259066 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139e93-25cf-4e62-95bf-1a3179725702 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.259869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139e93-25cf-4e62-95bf-1a3179725702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.264149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc56da0-260b-4ff9-b533-19b09601e7fc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.266317 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.266962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.267245 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.267520 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.268167 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.270153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.280387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a5139e93-25cf-4e62-95bf-1a3179725702 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:40.281552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a5139e93-25cf-4e62-95bf-1a3179725702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:40.289826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1322/2652] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:40.298965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.299263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.299502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.321840 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:40.323081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:40.330929 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1323/2653] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:40.335036 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.344123 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d109c-5e50-4fc5-ac42-1729573de8ad None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.348688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.349297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.349592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.355626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.355626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:40.355626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.355626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.363720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1324/2654] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:40.368491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.378132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd917f00-288e-4a4d-8203-28653534a0a1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.378782 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1331/2655] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:40.383203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.384532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.384835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:40.385135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.385481 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.386328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.387054 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.389006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.397077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992589b-0853-4bd6-863f-7fbebf23c18d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.401853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.402482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.402794 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.403096 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.403742 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.405340 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.415414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1332/2656] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:40.419565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.429960 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65663-868c-4587-a643-40cde3306894 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.434508 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.435169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.435448 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.435719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] PATH_INFO: `/v3/projects/6f6b55c63697440c8c4ab180d96c1a37/users/46a7988d59c5404abb130a38c3663df1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.450633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6b55c63697440c8c4ab180d96c1a37, user_id=46a7988d59c5404abb130a38c3663df1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.453262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.478087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1325/2657] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:40.482252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.549593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a0cf8-7d77-4cc4-9f2b-53d88a1a010f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.549932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1333/2658] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:40 2022] PUT /identity/v3/projects/6f6b55c63697440c8c4ab180d96c1a37/users/46a7988d59c5404abb130a38c3663df1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:40.549932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.595363 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:40.617596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054697-09e9-4097-959d-1b4fff02055d admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.618787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.619425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.619738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.620013 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.620653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.623097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.623301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.624159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.624629 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.625498 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.626151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.627781 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.638752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1334/2659] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:40.643089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.653172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c460e96-a786-4de5-af6f-7e36fd261c79 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.658633 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.668506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.668506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.668506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] PATH_INFO: `/v3/projects/6f6b55c63697440c8c4ab180d96c1a37/users/46a7988d59c5404abb130a38c3663df1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.668506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6b55c63697440c8c4ab180d96c1a37, user_id=46a7988d59c5404abb130a38c3663df1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.668506 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8b28a-162e-4725-a05c-37801c8442f3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.676134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1335/2660] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:40 2022] PUT /identity/v3/projects/6f6b55c63697440c8c4ab180d96c1a37/users/46a7988d59c5404abb130a38c3663df1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:40.689093 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.689404 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.689649 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.722548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] MFA Rules not processed for user `46a7988d59c5404abb130a38c3663df1`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:40.723276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1326/2661] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:40.723814 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:40.728864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.751948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8862-64a0-4e1a-8363-40aa89bba083 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.791830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:40.816334 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1336/2662] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:40.819635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.820259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.820533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.820805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.821467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.823264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.828022 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.830527 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:40.835262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1327/2663] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:40.840369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.854933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe59a-0996-4e69-847d-942b0593625b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.861748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.862364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.862639 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.862947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.863590 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.865285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.875940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1328/2664] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:40.880282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:40.890808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f048-dfb3-4d2f-b46e-922906b73b17 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.896577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.897823 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.898122 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.898373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] PATH_INFO: `/v3/projects/7d01b966762c46f28c83c9cc4e8cb33e/users/61922d7e04ed4c69bafcc5c2855539f8/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.920588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d01b966762c46f28c83c9cc4e8cb33e, user_id=61922d7e04ed4c69bafcc5c2855539f8, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.926146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd62430-bb4d-44be-8235-a7c7da81012d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:40.926407 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae665967-f4d5-4646-8831-f8ee6bf1b71e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.929977 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:40.930630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:40.930946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:40.931260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:40.941268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:40.942835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:40.953821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1337/2665] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:41.012467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:41.014503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:41.026563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1329/2666] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:40 2022] PUT /identity/v3/projects/7d01b966762c46f28c83c9cc4e8cb33e/users/61922d7e04ed4c69bafcc5c2855539f8/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:41.034979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8ecb2-6b20-4858-b9cc-c9fa7237b204 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:41.035222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:41.035222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:41.035222 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:41.057297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:41.058680 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:41.095327 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:41.096299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:41.096591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:41.097219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:41.100969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:41.108247 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:41.133219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1330/2667] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:41.143563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:41.144889 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:41.166562 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:41.168163 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:41.182018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1338/2668] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:41.213027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053944e-1497-489e-8ac9-91d498ea0319 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:41.216768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:41.217385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:41.217854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:41.218027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:41.229717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:41.231213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e831-5c46-4512-9a26-973d72dafeea service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:41.241832 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1331/2669] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:46.890844 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:46.904066 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12440f-6824-46fc-986c-f5ebb9597ae0 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:46.921432 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:46.922830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:46.932918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:46.932918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] PATH_INFO: `/v3/users/4185c6d71815472d96cb9ecd044073e0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:46.932918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4185c6d71815472d96cb9ecd044073e0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:46.932918 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:47.207411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4185c6d71815472d96cb9ecd044073e0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:47.208241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4185c6d71815472d96cb9ecd044073e0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:47.245309 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab194b-f190-4141-872a-772e512d9817 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.245867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab194b-f190-4141-872a-772e512d9817 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.245867 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab194b-f190-4141-872a-772e512d9817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.279382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-08ab194b-f190-4141-872a-772e512d9817 None None] MFA Rules not processed for user `61922d7e04ed4c69bafcc5c2855539f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:47.280808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-08ab194b-f190-4141-872a-772e512d9817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:47.298885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4185c6d71815472d96cb9ecd044073e0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:47.310409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4185c6d71815472d96cb9ecd044073e0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:47.321968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:47.333235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-16059da7-fd68-4de9-8ff2-5770335fca31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4185c6d71815472d96cb9ecd044073e0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:47.347337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1339/2670] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:46 2022] DELETE /identity/v3/users/4185c6d71815472d96cb9ecd044073e0 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:47.351173 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.354713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.354713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.376452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:47.377780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:47.379666 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1332/2671] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:47.383473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab194b-f190-4141-872a-772e512d9817 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:47.383473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-08ab194b-f190-4141-872a-772e512d9817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:47.405712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab194b-f190-4141-872a-772e512d9817 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:47.462040 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1340/2672] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:47.471834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:47.473801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:47.500222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:47.502825 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:47.507501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.507968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.508553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.509751 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:47.513720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:47.587993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb381-c49a-47db-ab0d-84f091da1e8c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:47.592521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:47.594465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.595473 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.595873 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.609481 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:47.612971 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:47.630464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1341/2673] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:47.701826 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1333/2674] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:47 2022] POST /identity/v3/projects => generated 385 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:47.706465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:47.733546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d10e7d-bec4-4fc3-b21f-e41026be44ec service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:47.816728 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:47.817456 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.817801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.818149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.819472 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:47.821719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:47.934268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1342/2675] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:47 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:47.939196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:47.952773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1a0b-8f93-48db-880d-81dd50aee662 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:47.957790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:47.958817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.963512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.963512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:47.963512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:47.963781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:47.982218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1334/2676] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:47.982463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:47.993459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b2b5-8f1e-4629-a7f9-956a143da989 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:47.998380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:47.999100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:47.999384 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:47.999746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] PATH_INFO: `/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf/users/bbd8f285fa43408187027953fdbff515/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:48.020986 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec7262ec3104e37a1f6eed28bf4c4cf, user_id=bbd8f285fa43408187027953fdbff515, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:48.032139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:48.140833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1343/2677] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:47 2022] PUT /identity/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf/users/bbd8f285fa43408187027953fdbff515/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:48.144740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:48.155483 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7117d-987a-486a-9dab-eec9139ce530 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:48.241310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:48.241910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:48.242263 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:48.242543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:48.243268 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:48.244981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:48.256348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1335/2678] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:48.260834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:48.276159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090df2c-c97b-4368-b5ba-2d83e429752c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:48.283560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:48.284342 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:48.284698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:48.285045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] PATH_INFO: `/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf/users/bbd8f285fa43408187027953fdbff515/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:48.294779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec7262ec3104e37a1f6eed28bf4c4cf, user_id=bbd8f285fa43408187027953fdbff515, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:48.298594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:48.315998 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1344/2679] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:48 2022] PUT /identity/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf/users/bbd8f285fa43408187027953fdbff515/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:48.519568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:48.524739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:48.542624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12667c01-8af5-4ce2-bbd1-eff48b803dd1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:48.607737 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:48.627067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:48.627719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:48.628021 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:48.628436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] PATH_INFO: `/v3/projects/455d7682d283471e9f53e62b610c036f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:48.629532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] RBAC: Authorizing `identity:delete_project(project_id=455d7682d283471e9f53e62b610c036f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:48.631528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:48.773000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455d7682d283471e9f53e62b610c036f', 'request_id': 'req-7811d895-c3ea-467a-91ac-2d7f34f5316e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:48.780931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455d7682d283471e9f53e62b610c036f', 'request_id': 'req-7811d895-c3ea-467a-91ac-2d7f34f5316e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:48.841401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455d7682d283471e9f53e62b610c036f', 'request_id': 'req-7811d895-c3ea-467a-91ac-2d7f34f5316e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:48.854285 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] The token cache is being invalidate because project 455d7682d283471e9f53e62b610c036f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:48.854572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:48.855278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:48.856828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1336/2680] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:48 2022] DELETE /identity/v3/projects/455d7682d283471e9f53e62b610c036f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:48.861211 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:48.862972 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:48.877542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dac7c3-6cd3-40ce-8e28-fe072a71abb7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:48.939400 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:48.940892 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:48.941128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:48.941565 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] PATH_INFO: `/v3/users/46a7988d59c5404abb130a38c3663df1` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:48.942988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a7988d59c5404abb130a38c3663df1)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:48.945861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:48.967024 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:49.157851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a7988d59c5404abb130a38c3663df1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:49.159025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a7988d59c5404abb130a38c3663df1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:49.269808 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a7988d59c5404abb130a38c3663df1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:49.278344 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a7988d59c5404abb130a38c3663df1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:49.310821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-423a3dcc-39eb-42bc-8380-dfec19ca9c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a7988d59c5404abb130a38c3663df1'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:49.317199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1345/2681] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:48 2022] DELETE /identity/v3/users/46a7988d59c5404abb130a38c3663df1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:49.787135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:49.807544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d895-c3ea-467a-91ac-2d7f34f5316e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:49.876032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:49.876715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:49.877015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:49.877295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] PATH_INFO: `/v3/projects/6f6b55c63697440c8c4ab180d96c1a37` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:49.878497 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6b55c63697440c8c4ab180d96c1a37)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:49.880675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:50.116073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6b55c63697440c8c4ab180d96c1a37', 'request_id': 'req-fa693a06-7ed9-4444-a225-1cef1b8b526a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:50.128378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6b55c63697440c8c4ab180d96c1a37', 'request_id': 'req-fa693a06-7ed9-4444-a225-1cef1b8b526a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:50.224532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6b55c63697440c8c4ab180d96c1a37', 'request_id': 'req-fa693a06-7ed9-4444-a225-1cef1b8b526a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:50.246423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] The token cache is being invalidate because project 6f6b55c63697440c8c4ab180d96c1a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:50.246915 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:50.247847 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fa693a06-7ed9-4444-a225-1cef1b8b526a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:50.250146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1337/2682] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:49 2022] DELETE /identity/v3/projects/6f6b55c63697440c8c4ab180d96c1a37 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:50.255885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.255885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.256555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.295270 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:50.298950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:50.426287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1346/2683] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:50.435657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.435657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.435657 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.444255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:50.459358 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:50.460833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:50.470273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1338/2684] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:50.480179 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:50.496546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786ade-9ae9-4e52-8111-55dc6e0bb48a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:50.505002 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:50.505659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.506767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.507064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.509469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:50.510267 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:50.511702 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:50.538930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1347/2685] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:50.540145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:50.553901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2f86b-699b-4381-a3df-b6070897ef56 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:50.567351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:50.568005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.568259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.568558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.569198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:50.571258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:50.703960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1339/2686] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:50 2022] POST /identity/v3/projects => generated 409 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:50.709048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:50.723439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aa916-ccb8-41a6-8f29-26ae76c01bd7 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:50.800198 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:50.800985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.801271 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.801840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.802512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:50.804659 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:50.969587 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1348/2687] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:50 2022] POST /identity/v3/users => generated 305 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:50.975422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:50.987865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187478-730b-4c1d-a266-9c1404b8ad24 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:50.998874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:50.999423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.003299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.003299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.003299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:51.003299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:51.020618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1340/2688] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:51.025591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:51.038599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e25e6-f07c-4d81-ac4f-88c6905153fc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:51.044315 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:51.045463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.045857 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.046201 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.047265 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:51.049122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60761-7561-4b83-80ee-84bb4bbdc88e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:51.056444 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:51.064624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1349/2689] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:51.069629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:51.084772 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419b70e-1a4f-4333-8c8c-ee4d5123915c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:51.097334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:51.098148 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.098526 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.098950 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] PATH_INFO: `/v3/projects/06a86faa7928446aa7bf90946c614378/users/409cf3d329f946179044a1f07b4e3b7a/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.135912 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a86faa7928446aa7bf90946c614378, user_id=409cf3d329f946179044a1f07b4e3b7a, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:51.138981 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:51.401384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1341/2690] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:51 2022] PUT /identity/v3/projects/06a86faa7928446aa7bf90946c614378/users/409cf3d329f946179044a1f07b4e3b7a/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:51.411889 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.412541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.412541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.450159 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] MFA Rules not processed for user `409cf3d329f946179044a1f07b4e3b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:51.451464 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:51.531392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1350/2691] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:51.536016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:51.621768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8078-ec81-4780-a222-3a087a51d5b7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:51.633060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:51.633787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.634143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.634786 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.635544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:51.639921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:51.757592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1342/2692] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:51 2022] POST /identity/v3/projects => generated 409 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:51.759237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:51.785592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d943c7-e0e3-40d0-936b-204a17acce50 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:51.898551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:51.898551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:51.898551 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:51.898993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:51.903424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:51.924946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.025849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1351/2693] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:51 2022] POST /identity/v3/users => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:52.030385 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:52.046766 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4028dce-56c5-4074-a194-562704f92f3c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:52.054857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:52.057866 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.058398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.058914 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.064940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:52.064940 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.076858 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1343/2694] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:52.081053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:52.098009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28bdc2-e51f-44af-a038-2b7ce93043e2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:52.104306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:52.105069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.105441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.105834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] PATH_INFO: `/v3/projects/86d0423182144fd9a49db2b844d3fdc0/users/cb3b83b081794f7994b33abdfa15309b/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.123504 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d0423182144fd9a49db2b844d3fdc0, user_id=cb3b83b081794f7994b33abdfa15309b, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:52.126544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.212513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1352/2695] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:52 2022] PUT /identity/v3/projects/86d0423182144fd9a49db2b844d3fdc0/users/cb3b83b081794f7994b33abdfa15309b/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:52.218610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:52.235343 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93733-7f54-4516-840d-0b90685e5a74 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:52.307041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:52.307618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.307900 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.308192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.308830 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:52.310582 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405e705-be61-41c0-a4f3-582d9c1a803a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.322097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1344/2696] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:52.326501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:52.338351 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c21a-de65-4f7c-a09a-1b457cb23466 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:52.360801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:52.361651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.361988 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.362276 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] PATH_INFO: `/v3/projects/86d0423182144fd9a49db2b844d3fdc0/users/cb3b83b081794f7994b33abdfa15309b/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.364684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d0423182144fd9a49db2b844d3fdc0, user_id=cb3b83b081794f7994b33abdfa15309b, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:52.369328 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.383675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1353/2697] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:52 2022] PUT /identity/v3/projects/86d0423182144fd9a49db2b844d3fdc0/users/cb3b83b081794f7994b33abdfa15309b/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:52.397242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.397242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.397242 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.418921 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:52.428518 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] MFA Rules not processed for user `cb3b83b081794f7994b33abdfa15309b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:52.434074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:52.521492 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1345/2698] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:52 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:52.534279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:52.534279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:52.597229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:52.612107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43fd7f-afcf-4d14-9718-7543d9db7c8e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:52.616758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:52.617718 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:52.618279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:52.618833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:52.633143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:52.634946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54af392-abdb-4a8e-a859-36e1b1e24a7a service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:52.648226 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1354/2699] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:52 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:53.138401 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:53.153502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0541e1-fcac-4b83-8baf-1e9a83616591 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:53.157937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:53.159311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:53.159899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:53.160272 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:53.241524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:53.245734 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:53.253154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1346/2700] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:54.451443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2ecf-2b0a-4598-b73a-a8ff5248dffb None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:54.451944 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2ecf-2b0a-4598-b73a-a8ff5248dffb None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:54.452148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2ecf-2b0a-4598-b73a-a8ff5248dffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:54.480350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-096c2ecf-2b0a-4598-b73a-a8ff5248dffb None None] MFA Rules not processed for user `bbd8f285fa43408187027953fdbff515`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:54.481583 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-096c2ecf-2b0a-4598-b73a-a8ff5248dffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:54.564606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1355/2701] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:04:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:54.578817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:54.580369 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:54.644909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08f07-e6b9-4af3-ad5a-6cb1708d6372 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:54.649144 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:54.649757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:54.650042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:54.650337 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:54.652799 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:54.701617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:54.727306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:54.740259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:54.740259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1347/2702] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:55.063934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:55.064396 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:55.064747 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:55.104427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:55.105639 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:55.184684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1356/2703] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:04:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:55.327300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:55.328576 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:55.420795 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70218a0-33dc-45e4-bfe4-0e41203e97e0 service glance] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:55.440729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:55.443787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:55.443787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:55.443787 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:55.460693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:55.462271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c05a7-df81-4714-abe1-ebe5ca45aa27 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:55.472334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1348/2704] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:04:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:05:04.153222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:04.234084 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f27a5-f0d6-435c-b6b9-d3cd679df4b5 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:04.239469 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:04.240122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:04.240442 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:04.240721 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] PATH_INFO: `/v3/users/409cf3d329f946179044a1f07b4e3b7a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:04.241888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409cf3d329f946179044a1f07b4e3b7a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:04.243917 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:04.252848 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:04.496382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409cf3d329f946179044a1f07b4e3b7a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:04.497155 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409cf3d329f946179044a1f07b4e3b7a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:04.613065 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409cf3d329f946179044a1f07b4e3b7a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:04.620979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409cf3d329f946179044a1f07b4e3b7a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:04.644948 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409cf3d329f946179044a1f07b4e3b7a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:04.660067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1357/2705] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:04 2022] DELETE /identity/v3/users/409cf3d329f946179044a1f07b4e3b7a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:04.662332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:04.667519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:04.667519 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:04.686060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:04.687320 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:04.772855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1349/2706] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:04.783858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:04.785522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:04.856419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46fa41-5adb-45ca-bcf0-a49d9eaf1986 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:04.860218 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:04.860851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:04.861134 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:04.861411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:04.872411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:04.873910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:04.888495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1358/2707] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:05.562403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:05.565032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:05.599015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea8ca8-d212-4393-891c-e4be4581a841 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:05.611204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:05.612564 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:05.613676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:05.613676 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] PATH_INFO: `/v3/projects/06a86faa7928446aa7bf90946c614378` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:05.615467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a86faa7928446aa7bf90946c614378)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:05.619465 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:05.929429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a86faa7928446aa7bf90946c614378', 'request_id': 'req-10ae8aee-16d8-4918-899f-81cbd2d5954d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:05.937467 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a86faa7928446aa7bf90946c614378', 'request_id': 'req-10ae8aee-16d8-4918-899f-81cbd2d5954d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.008142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a86faa7928446aa7bf90946c614378', 'request_id': 'req-10ae8aee-16d8-4918-899f-81cbd2d5954d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.017154 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:06.021188 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] The token cache is being invalidate because project 06a86faa7928446aa7bf90946c614378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:06.022217 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.022824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-10ae8aee-16d8-4918-899f-81cbd2d5954d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.025798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1350/2708] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:05 2022] DELETE /identity/v3/projects/06a86faa7928446aa7bf90946c614378 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:06.029989 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:06.039768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d34e77-7eef-4dc7-94cd-91d04a819c22 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:06.101399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:06.102025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:06.102309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:06.102579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] PATH_INFO: `/v3/users/cb3b83b081794f7994b33abdfa15309b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:06.103729 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3b83b081794f7994b33abdfa15309b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:06.105650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:06.306467 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:06.321676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3b83b081794f7994b33abdfa15309b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.322553 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3b83b081794f7994b33abdfa15309b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.328730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:06.329016 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:06.329316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:06.350087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] MFA Rules not processed for user `61922d7e04ed4c69bafcc5c2855539f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:06.351363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:06.396069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3b83b081794f7994b33abdfa15309b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.404254 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3b83b081794f7994b33abdfa15309b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.408971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1351/2709] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:06.418862 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-9fc11c39-6a88-449d-b4d9-487ff82f577c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3b83b081794f7994b33abdfa15309b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.426172 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1359/2710] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:06 2022] DELETE /identity/v3/users/cb3b83b081794f7994b33abdfa15309b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:06.955258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:06.956759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:06.976568 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6efcb-d9b5-43b6-95a2-80de2bdc9bd2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.042630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.043299 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.043575 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.043844 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] PATH_INFO: `/v3/projects/86d0423182144fd9a49db2b844d3fdc0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.045041 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d0423182144fd9a49db2b844d3fdc0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.047024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.193359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d0423182144fd9a49db2b844d3fdc0', 'request_id': 'req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.200063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d0423182144fd9a49db2b844d3fdc0', 'request_id': 'req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.293702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d0423182144fd9a49db2b844d3fdc0', 'request_id': 'req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.307069 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] The token cache is being invalidate because project 86d0423182144fd9a49db2b844d3fdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:07.307437 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.308155 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-511b2a9d-7f1a-43d7-a92f-a3d9b6d09c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.309705 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1352/2711] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:06 2022] DELETE /identity/v3/projects/86d0423182144fd9a49db2b844d3fdc0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:07.315210 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.315658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.315658 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.336069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:07.337283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:07.397621 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1360/2712] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:07.406547 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.406908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.407177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.427699 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:07.428998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:07.437037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1353/2713] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:07.441521 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.451136 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc2e0-0c88-4171-88c6-d88812f7c6b9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.455630 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.456241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.456537 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.456809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.457457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:05:07.457757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.459984 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.471339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1361/2714] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:05:07.476718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.486085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fbd6e-7180-408e-a9dc-17861b3d8575 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.490757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.491364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.491729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.492005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.492641 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.494548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.641817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1354/2715] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:05:07.643214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.654612 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee5077-7d87-4c98-9913-835316f1bb0b admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.725143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.725937 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.726222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.726493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.727186 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.729137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.842581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1362/2716] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:07 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:07.846895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.857543 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905a277-9c4c-4a20-b3c4-07bd594c1ed9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.862240 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.862967 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.863172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.863444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.864098 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.866223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.867761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:07.877461 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1355/2717] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:07.882042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.892674 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9af55-66d9-4e68-b7c6-0bece83b78c2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.897557 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.898195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.898501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.898768 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.899427 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.901049 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca3b5-088c-4af9-8584-7be9a63188e0 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.912758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1363/2718] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:07.921717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.928569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3d53-3959-4e47-8a8b-97c96091e53c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.933314 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.934004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.934286 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.934561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] PATH_INFO: `/v3/projects/96c5f875dab04f969ded5ecfd72e1763/users/95bede5f8e654598b07729bdcc3a071b/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.951330 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c5f875dab04f969ded5ecfd72e1763, user_id=95bede5f8e654598b07729bdcc3a071b, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.954073 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:08.054735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1356/2719] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:07 2022] PUT /identity/v3/projects/96c5f875dab04f969ded5ecfd72e1763/users/95bede5f8e654598b07729bdcc3a071b/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:08.066640 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9006f-fa90-428c-96e7-82179709c774 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.066807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9006f-fa90-428c-96e7-82179709c774 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.067094 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9006f-fa90-428c-96e7-82179709c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.096287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:08.099696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9dd9006f-fa90-428c-96e7-82179709c774 None None] MFA Rules not processed for user `95bede5f8e654598b07729bdcc3a071b`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:08.100924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9dd9006f-fa90-428c-96e7-82179709c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:08.189701 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1364/2720] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:08.200255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:08.202213 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:08.269198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52b1-409a-4177-8054-c8832b592a2b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:08.273269 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:08.273879 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.274304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.274577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.283668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:08.285132 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:08.295620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1357/2721] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:10.261208 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.261685 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.261802 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.283967 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:10.285205 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:10.358330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1365/2722] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Aug 24 09:05:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:10.377076 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.380147 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:10.456755 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab898282-c868-4c96-aaae-005bfcf1bc60 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.474128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.475507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.475801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.476101 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.487476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.489962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86000a-f28e-4de2-bcec-d2c18186b020 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.500985 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1358/2723] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:05:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:05:20.351581 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=51 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:20.495871 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:37.028077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:37.119950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53c20b-8e21-4a9d-b891-49ae6eccfa1d None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:37.125146 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:37.126148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.126606 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.127256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] PATH_INFO: `/v3/users/95bede5f8e654598b07729bdcc3a071b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.135725 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95bede5f8e654598b07729bdcc3a071b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:37.137891 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:37.379068 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95bede5f8e654598b07729bdcc3a071b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.379756 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95bede5f8e654598b07729bdcc3a071b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.470696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95bede5f8e654598b07729bdcc3a071b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.499023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95bede5f8e654598b07729bdcc3a071b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.505569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95bede5f8e654598b07729bdcc3a071b'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.513753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1366/2724] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:37 2022] DELETE /identity/v3/users/95bede5f8e654598b07729bdcc3a071b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:37.519177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.519177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.519177 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.541803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:37.544413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.626022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1359/2725] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:37.636752 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:37.638830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.731617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7a819-f844-4978-885e-1d899bd0ef36 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:37.736115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:37.736806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.737152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.737489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.752793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:37.752793 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:37.761617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1367/2726] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:38.223134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.225665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.254817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d424047-8195-4de1-a8f1-b6cdf8a3014f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.260480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.261250 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.261558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.261828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] PATH_INFO: `/v3/projects/96c5f875dab04f969ded5ecfd72e1763` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.263231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c5f875dab04f969ded5ecfd72e1763)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.265487 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.398395 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c5f875dab04f969ded5ecfd72e1763', 'request_id': 'req-8c125a55-68e7-4f16-9106-34881ce56415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.408162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c5f875dab04f969ded5ecfd72e1763', 'request_id': 'req-8c125a55-68e7-4f16-9106-34881ce56415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.499458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c5f875dab04f969ded5ecfd72e1763', 'request_id': 'req-8c125a55-68e7-4f16-9106-34881ce56415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.518619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] The token cache is being invalidate because project 96c5f875dab04f969ded5ecfd72e1763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:38.518974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.520302 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.535945 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1360/2727] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:38 2022] DELETE /identity/v3/projects/96c5f875dab04f969ded5ecfd72e1763 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:39.324364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:39.325840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:39.352959 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:39.383885 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cba519-e318-40aa-b068-cbe511e4235a service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:39.435465 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:39.436071 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:39.436354 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:39.436626 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:39.438401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:39.506380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:39.507828 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:39.517763 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1368/2728] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:42.646348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:42.648114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:42.712907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125a55-68e7-4f16-9106-34881ce56415 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:42.716886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:42.717509 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:42.717788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:42.718055 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:42.719908 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:42.772530 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:42.814196 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:42.817873 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:42.838910 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1361/2729] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:05:49.551586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=58 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:49.608195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:49.620018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:49.640484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273239a5-aeec-41c9-a4cb-e4102c59bdb0 service cinder] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:49.655945 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:49.657221 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:49.657788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:49.658502 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:49.664083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:49.847531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:49.849016 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:49.878513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1369/2730] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:49.883963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:49.903986 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d6733-c8fa-47d8-91f3-03bc3007ec23 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:49.911757 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:49.912798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:49.913341 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:49.913829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:49.918768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:49.976622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:49.979632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:50.002448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1362/2731] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:53.320117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:53.323237 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:53.340595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41460a-8128-4a00-9f7a-5cfa628af024 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:53.350958 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:53.353337 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:53.354115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:53.354716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] PATH_INFO: `/v3/users/a35fcaecf09947e78813f96ec93e6c0a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:53.356883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35fcaecf09947e78813f96ec93e6c0a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:53.360133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:53.593257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35fcaecf09947e78813f96ec93e6c0a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:53.594236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35fcaecf09947e78813f96ec93e6c0a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:53.673541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35fcaecf09947e78813f96ec93e6c0a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:53.692463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35fcaecf09947e78813f96ec93e6c0a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:53.723420 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35fcaecf09947e78813f96ec93e6c0a'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:53.737538 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1370/2732] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:53 2022] DELETE /identity/v3/users/a35fcaecf09947e78813f96ec93e6c0a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:54.269289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:54.281087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ee13f-773f-43c2-b5d3-4555a63d9bfb service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:54.374440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:54.375133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:54.375729 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:54.376180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] PATH_INFO: `/v3/projects/065b8b1394f24ce1973119ee8cd6bd2c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:54.377792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065b8b1394f24ce1973119ee8cd6bd2c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:54.380200 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:54.555608 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065b8b1394f24ce1973119ee8cd6bd2c', 'request_id': 'req-8b2d1deb-db7c-4b89-816d-ae5944200910'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:54.563677 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065b8b1394f24ce1973119ee8cd6bd2c', 'request_id': 'req-8b2d1deb-db7c-4b89-816d-ae5944200910'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:54.713894 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065b8b1394f24ce1973119ee8cd6bd2c', 'request_id': 'req-8b2d1deb-db7c-4b89-816d-ae5944200910'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:54.733597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] The token cache is being invalidate because project 065b8b1394f24ce1973119ee8cd6bd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:54.737220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:54.737220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:54.741622 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1363/2733] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:54 2022] DELETE /identity/v3/projects/065b8b1394f24ce1973119ee8cd6bd2c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:54.743755 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:54.746598 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:54.761999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b826-6452-4950-b5b1-1362a75e9bf8 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:54.842864 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:54.843107 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:54.843246 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:54.843510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] PATH_INFO: `/v3/users/e13368f0a82b42039f5e9a965d6afcab` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:54.844939 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13368f0a82b42039f5e9a965d6afcab)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:54.847051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:54.854398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:55.112609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13368f0a82b42039f5e9a965d6afcab'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:55.113309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13368f0a82b42039f5e9a965d6afcab'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:55.277148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13368f0a82b42039f5e9a965d6afcab'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:55.291798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13368f0a82b42039f5e9a965d6afcab'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:55.311516 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-dbd34c9e-4911-493a-b1f7-adee6f3309cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13368f0a82b42039f5e9a965d6afcab'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:55.319688 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1371/2734] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:54 2022] DELETE /identity/v3/users/e13368f0a82b42039f5e9a965d6afcab => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:55.921934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:55.924396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:55.941779 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d1deb-db7c-4b89-816d-ae5944200910 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:56.043733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:56.064522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:56.065402 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:56.066020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:56.066405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] PATH_INFO: `/v3/projects/5b1ec073f8634ba2927ea52b735f653c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:56.069448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1ec073f8634ba2927ea52b735f653c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:56.077957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:56.270361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1ec073f8634ba2927ea52b735f653c', 'request_id': 'req-6fe48249-17ed-45d3-988c-9cef8c6fa92a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:56.285640 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1ec073f8634ba2927ea52b735f653c', 'request_id': 'req-6fe48249-17ed-45d3-988c-9cef8c6fa92a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:56.392577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1ec073f8634ba2927ea52b735f653c', 'request_id': 'req-6fe48249-17ed-45d3-988c-9cef8c6fa92a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:56.431892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] The token cache is being invalidate because project 5b1ec073f8634ba2927ea52b735f653c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:56.432384 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:56.434597 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:56.436984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1364/2735] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:55 2022] DELETE /identity/v3/projects/5b1ec073f8634ba2927ea52b735f653c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:57.735439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:57.736055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:57.736055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:57.760425 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] MFA Rules not processed for user `8bec7c9367ee4c00a104cbe3753ee9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:57.761679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:57.842380 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1372/2736] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:02.556210 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:02.557922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:02.636868 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe48249-17ed-45d3-988c-9cef8c6fa92a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:02.641951 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:02.642611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:02.642925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:02.643206 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] PATH_INFO: `/v3/users/f1381dae2b774b48a590e21b3d30dbd3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:02.645189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1381dae2b774b48a590e21b3d30dbd3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:02.647340 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:02.807820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1381dae2b774b48a590e21b3d30dbd3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:02.912058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1381dae2b774b48a590e21b3d30dbd3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:02.923432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1381dae2b774b48a590e21b3d30dbd3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:02.924382 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1381dae2b774b48a590e21b3d30dbd3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:02.940660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1381dae2b774b48a590e21b3d30dbd3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:02.943445 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:02.949571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1365/2737] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:06:02 2022] DELETE /identity/v3/users/f1381dae2b774b48a590e21b3d30dbd3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:03.312979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:03.329403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e31057-7198-4dc8-819d-b1a5bb1a547a None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:03.399120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:03.399849 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:03.400140 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:03.400411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] PATH_INFO: `/v3/projects/12594f976e5f48d9926716737f86afe0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:03.401705 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] RBAC: Authorizing `identity:delete_project(project_id=12594f976e5f48d9926716737f86afe0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:03.403696 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:03.552304 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:03.564348 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12594f976e5f48d9926716737f86afe0', 'request_id': 'req-0b616939-9814-4388-ba75-4a138a3a58de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:03.570894 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12594f976e5f48d9926716737f86afe0', 'request_id': 'req-0b616939-9814-4388-ba75-4a138a3a58de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:03.670815 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12594f976e5f48d9926716737f86afe0', 'request_id': 'req-0b616939-9814-4388-ba75-4a138a3a58de'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:03.683357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] The token cache is being invalidate because project 12594f976e5f48d9926716737f86afe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:06:03.683686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:03.684408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:03.685991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1373/2738] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:06:03 2022] DELETE /identity/v3/projects/12594f976e5f48d9926716737f86afe0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:03.690689 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:03.692468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:03.708759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df6c0-1d45-4811-bafb-e5aef2dbd691 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:03.776848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:03.777501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:03.777777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:03.778047 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] PATH_INFO: `/v3/users/4987e06b1e6744f384e42591dc8174bd` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:03.779261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4987e06b1e6744f384e42591dc8174bd)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:03.781219 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:04.136373 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4987e06b1e6744f384e42591dc8174bd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.231463 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4987e06b1e6744f384e42591dc8174bd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.240388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4987e06b1e6744f384e42591dc8174bd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.241128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4987e06b1e6744f384e42591dc8174bd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.255319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4987e06b1e6744f384e42591dc8174bd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.262515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1366/2739] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:06:03 2022] DELETE /identity/v3/users/4987e06b1e6744f384e42591dc8174bd => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:06:04.598589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:04.600878 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:04.615682 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616939-9814-4388-ba75-4a138a3a58de admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:04.693310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:04.694030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:04.694316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:04.694590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] PATH_INFO: `/v3/projects/d579490e0cea488b97167ba13766fc47` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:04.696374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d579490e0cea488b97167ba13766fc47)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:04.698488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:04.826338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd579490e0cea488b97167ba13766fc47', 'request_id': 'req-0dd39e81-1f63-48cf-bc17-70da04c6040c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.833108 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd579490e0cea488b97167ba13766fc47', 'request_id': 'req-0dd39e81-1f63-48cf-bc17-70da04c6040c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.899667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd579490e0cea488b97167ba13766fc47', 'request_id': 'req-0dd39e81-1f63-48cf-bc17-70da04c6040c'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.911443 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] The token cache is being invalidate because project d579490e0cea488b97167ba13766fc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:06:04.911733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.912392 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:04.913865 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1374/2740] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:06:04 2022] DELETE /identity/v3/projects/d579490e0cea488b97167ba13766fc47 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:06.661032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:06.662629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:06.723763 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ddfd-be0d-4435-a30f-80bcf75f4935 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:06.757452 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:06.777350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:06.777993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:06.778396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:06.778738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:06.780658 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:06.856238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:06.857726 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:06.870598 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1367/2741] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:08.060372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:08.061934 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:08.085993 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39e81-1f63-48cf-bc17-70da04c6040c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:08.089736 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:08.090421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:08.090619 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:08.090924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:08.092800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:08.107120 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:08.108574 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:08.120535 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1375/2742] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:16.865554 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=38 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:18.114779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=69,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:59.117865 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:59.120199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:59.140652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2ca9c-bfcd-4634-a06e-954157d91997 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:59.157569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:59.158741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:59.158741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:59.158969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:59.162490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:59.184476 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:59.185431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826c205-3436-4dfb-8ebe-b9f2f8808aad service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:59.237768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1368/2743] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:59.568672 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:59.570719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:59.583811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f3b9-f397-46b9-8ae4-00241f5ff46f service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:59.595860 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:59.596493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:59.596776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:59.597058 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:59.599195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:59.610105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:59.611738 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:59.620936 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1376/2744] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:08.916261 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.916739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.916862 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.967183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] MFA Rules not processed for user `76a944cb3f8b4b639d9e4e5fceda0355`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:08.968405 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:09.030741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1369/2745] 10.222.0.64 () {58 vars in 967 bytes} [Wed Aug 24 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:07:09.071264 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.072673 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:09.161897 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14716d83-3fdf-48ef-94ed-15f92030de75 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.167010 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b614a59a2e40c09dc87e99a717caf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.167617 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.167901 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.168178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.176119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:09.177949 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:09.186099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1377/2746] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Aug 24 09:07:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:07:18.989610 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:18.991422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:19.073635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995f4de-d3d7-41d7-8bfe-56af8e6f2cd3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:19.078334 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:19.079023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.079406 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.079558 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] PATH_INFO: `/v3/users/61922d7e04ed4c69bafcc5c2855539f8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.081441 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61922d7e04ed4c69bafcc5c2855539f8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:19.083398 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:19.182938 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=47 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:19.293702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61922d7e04ed4c69bafcc5c2855539f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:19.375136 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61922d7e04ed4c69bafcc5c2855539f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:19.384017 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:19.387516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61922d7e04ed4c69bafcc5c2855539f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:19.388753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61922d7e04ed4c69bafcc5c2855539f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:19.409498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61922d7e04ed4c69bafcc5c2855539f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:19.421922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1370/2747] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:18 2022] DELETE /identity/v3/users/61922d7e04ed4c69bafcc5c2855539f8 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:19.716132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:19.731316 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdd3e9-5707-41a7-94e4-4cab8fc68061 service swift] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:19.765652 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:19.767645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:19.857452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:19.858234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.858600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.858995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] PATH_INFO: `/v3/projects/7d01b966762c46f28c83c9cc4e8cb33e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.860446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d01b966762c46f28c83c9cc4e8cb33e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:19.863283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:19.902586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac933e-d446-48aa-a256-4d9227a519a6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:19.917293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0880178d7ea842afba2a0a4b8bd6d842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:19.917293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.917293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.917293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.918095 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:19.941156 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:19.942925 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:19.962502 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1371/2748] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:20.027703 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d01b966762c46f28c83c9cc4e8cb33e', 'request_id': 'req-7ebfdab0-10ff-423e-a6d4-55c8dc179192'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.049966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d01b966762c46f28c83c9cc4e8cb33e', 'request_id': 'req-7ebfdab0-10ff-423e-a6d4-55c8dc179192'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.164544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d01b966762c46f28c83c9cc4e8cb33e', 'request_id': 'req-7ebfdab0-10ff-423e-a6d4-55c8dc179192'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.179776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] The token cache is being invalidate because project 7d01b966762c46f28c83c9cc4e8cb33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:20.180079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.180778 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.182350 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1378/2749] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:19 2022] DELETE /identity/v3/projects/7d01b966762c46f28c83c9cc4e8cb33e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:20.186727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.188528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.202230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6546e-afd1-440c-9a3b-a3170124f55b service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.266022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.266573 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.266882 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.267600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] PATH_INFO: `/v3/users/bbd8f285fa43408187027953fdbff515` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.268802 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd8f285fa43408187027953fdbff515)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:20.270817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:20.478841 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd8f285fa43408187027953fdbff515'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.632399 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd8f285fa43408187027953fdbff515'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.644015 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd8f285fa43408187027953fdbff515'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.649089 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd8f285fa43408187027953fdbff515'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.649859 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:20.661506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd8f285fa43408187027953fdbff515'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:20.667781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1372/2750] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:20 2022] DELETE /identity/v3/users/bbd8f285fa43408187027953fdbff515 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:20.950950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.953152 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.966605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfdab0-10ff-423e-a6d4-55c8dc179192 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.037992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.038573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.038913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.039206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] PATH_INFO: `/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.040355 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec7262ec3104e37a1f6eed28bf4c4cf)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.042569 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.146812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.159095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec7262ec3104e37a1f6eed28bf4c4cf', 'request_id': 'req-fb56b235-1d5a-49cd-9991-6050e9d7fc45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.164514 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec7262ec3104e37a1f6eed28bf4c4cf', 'request_id': 'req-fb56b235-1d5a-49cd-9991-6050e9d7fc45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.249680 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec7262ec3104e37a1f6eed28bf4c4cf', 'request_id': 'req-fb56b235-1d5a-49cd-9991-6050e9d7fc45'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.261819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] The token cache is being invalidate because project 0ec7262ec3104e37a1f6eed28bf4c4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:21.262175 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.262876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.278888 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1379/2751] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:20 2022] DELETE /identity/v3/projects/0ec7262ec3104e37a1f6eed28bf4c4cf => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:27.600957 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.602528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:27.671174 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6b059-1923-4284-a744-d20a29b604d6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.740306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.741049 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.741448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.741828 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.744000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:27.836718 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.838193 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.850092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1373/2752] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:29.876647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.878281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:29.951048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b235-1d5a-49cd-9991-6050e9d7fc45 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.957536 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.958401 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.958692 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.958966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.961523 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:29.975430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.976957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.987785 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1380/2753] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:30.037349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.040559 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:30.056759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6506d-a98e-4578-ac9a-f072459da417 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.064026 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.065153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.065636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.066128 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] PATH_INFO: `/v3/users/8bec7c9367ee4c00a104cbe3753ee9ce` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.068231 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bec7c9367ee4c00a104cbe3753ee9ce)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.072078 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.267548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.270887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bec7c9367ee4c00a104cbe3753ee9ce'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.363695 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bec7c9367ee4c00a104cbe3753ee9ce'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.371752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bec7c9367ee4c00a104cbe3753ee9ce'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.372458 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bec7c9367ee4c00a104cbe3753ee9ce'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.386103 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-75c57192-bfc8-4cc8-bb09-09ee8c7e46cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bec7c9367ee4c00a104cbe3753ee9ce'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.393923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1374/2754] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:30 2022] DELETE /identity/v3/users/8bec7c9367ee4c00a104cbe3753ee9ce => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:30.732666 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.748151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e844cb-4801-4354-8e7a-2b78ef2486b5 service placement] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.821178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.821741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.822020 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.822293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] PATH_INFO: `/v3/projects/4fab402e38f1412b8c6b895788d11a28` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.823449 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fab402e38f1412b8c6b895788d11a28)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.825338 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.931507 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fab402e38f1412b8c6b895788d11a28', 'request_id': 'req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:30.933476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.938157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fab402e38f1412b8c6b895788d11a28', 'request_id': 'req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:31.021286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fab402e38f1412b8c6b895788d11a28', 'request_id': 'req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:31.033951 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] The token cache is being invalidate because project 4fab402e38f1412b8c6b895788d11a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:31.034257 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:31.034957 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-4708e006-50a0-4b3e-a20f-0a7c0caf41b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:31.036501 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1381/2755] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:30 2022] DELETE /identity/v3/projects/4fab402e38f1412b8c6b895788d11a28 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:31.041468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.042039 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.042202 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.063475 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:31.064997 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.129005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1375/2756] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:07:31.137804 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.138085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.138360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.158169 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:31.159908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.167751 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1382/2757] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:31.172146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.183965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae7202-82fd-47a2-af68-03ec15cab910 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.188606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.189218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.189505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.189778 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.190432 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:31.190839 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.192534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.204534 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1376/2758] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:31.209421 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.218779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3aa7b0-764c-4805-bcc7-a2e5ba316b6b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.224450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.225067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.225346 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.225644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.226297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.228244 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.337479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1383/2759] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:31.347353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.357544 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700eef-9b76-476a-8059-dff2353a2b86 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.416392 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.416999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.417282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.417552 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.418184 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.420235 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.530495 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1377/2760] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:31.535851 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.546359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f369-c117-4fed-8abf-aeada9502062 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.551203 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.551803 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.552089 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.552365 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.553036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.555075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.565531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1384/2761] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:31.569781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.580053 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7fde-3763-4a0a-807e-1309a9b0e657 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.585466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.586074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.586353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.586626 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.587304 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.588904 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9260fdb-7d00-438a-96cf-05f85290d38e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.600396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1378/2762] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:31.604801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.615124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8475-f41d-4421-aa89-c8af23b778b5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.619816 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.620468 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.620753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.621028 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] PATH_INFO: `/v3/projects/7fd336bcff54481da861882d35a6e091/users/e57bbf3357cf4767b4989f0965400308/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.637841 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd336bcff54481da861882d35a6e091, user_id=e57bbf3357cf4767b4989f0965400308, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.640624 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.689366 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1385/2763] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:31 2022] PUT /identity/v3/projects/7fd336bcff54481da861882d35a6e091/users/e57bbf3357cf4767b4989f0965400308/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:31.700490 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.702325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.702325 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.729368 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] MFA Rules not processed for user `e57bbf3357cf4767b4989f0965400308`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:31.730606 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.743102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:31.797365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1379/2764] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:31.809165 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.811799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.881423 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6912d-cc5e-44d4-a9a3-57fcd1067e96 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.885411 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.886025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.886307 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.886579 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.896519 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.898000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628f2e-ad8f-4aab-855f-93a32cd07b6f service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.908515 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1386/2765] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:41.902992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=63,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:41.923688 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:41.986393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a155435-e4a5-4a80-9fe3-1fb38216ad1f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:41.991142 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:41.991775 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:41.992058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:41.992332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] PATH_INFO: `/v3/users/e57bbf3357cf4767b4989f0965400308` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:41.993479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57bbf3357cf4767b4989f0965400308)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:41.995478 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:42.214561 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57bbf3357cf4767b4989f0965400308'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:42.346693 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57bbf3357cf4767b4989f0965400308'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:42.355160 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57bbf3357cf4767b4989f0965400308'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:42.362632 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57bbf3357cf4767b4989f0965400308'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:42.368592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57bbf3357cf4767b4989f0965400308'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:42.374727 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1380/2766] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:41 2022] DELETE /identity/v3/users/e57bbf3357cf4767b4989f0965400308 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:42.379859 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.380252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.380543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.400144 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:42.401357 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:42.461397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1387/2767] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:42.471548 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:42.473088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:42.515347 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:42.532555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0c2e-b670-49ae-9c02-72cd45ae8527 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:42.536354 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:42.536968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.537247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.537528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.548254 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:42.557854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e7ec1-4e53-47bf-b429-8c1ef79c01cd service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:42.559976 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1381/2768] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:43.017139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.018596 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.031290 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23df64-5c7a-4c10-b711-495723851c66 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.036776 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.037460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.037743 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.038013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] PATH_INFO: `/v3/projects/7fd336bcff54481da861882d35a6e091` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.039174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd336bcff54481da861882d35a6e091)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.041168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.159026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd336bcff54481da861882d35a6e091', 'request_id': 'req-8f846025-c06a-4781-845d-fcf863fe33c6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:43.165050 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd336bcff54481da861882d35a6e091', 'request_id': 'req-8f846025-c06a-4781-845d-fcf863fe33c6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:43.255616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd336bcff54481da861882d35a6e091', 'request_id': 'req-8f846025-c06a-4781-845d-fcf863fe33c6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:43.267479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] The token cache is being invalidate because project 7fd336bcff54481da861882d35a6e091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:43.267697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:43.268333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-8f846025-c06a-4781-845d-fcf863fe33c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:43.270330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1388/2769] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:43 2022] DELETE /identity/v3/projects/7fd336bcff54481da861882d35a6e091 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:43.275571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.278075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.278075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.295665 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:43.296876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.354515 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1382/2770] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:43.362399 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.362719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.367117 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.386950 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:43.388147 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.396966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1389/2771] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:43.401260 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.411947 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ebd5-fc5a-45e1-bde7-bef266b748c4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.416546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.417138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.417415 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.417946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.418624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:43.419028 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.420623 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.432429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1383/2772] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:43.436713 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.445767 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fb039-18be-44c1-8c38-020c0ecc2231 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.450359 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.450994 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.451273 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.451546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.452174 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.454187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.577099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1390/2773] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:43.581362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.592137 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecbbe5-d29b-4667-950b-438e7f7c799c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.647853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.648468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.648749 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.649025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.649660 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.651612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.713571 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1384/2774] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/users => generated 279 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:43.719575 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.730824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3a359-7348-471e-99ab-66ac226768a9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.735410 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.736012 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.736297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.736573 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.737215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.738883 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.748664 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1391/2775] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:43.752741 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.763630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a31e19-31a8-48a8-b529-00b7b708807d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.768265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.775656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.775656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.775656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.775656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.775656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750667a8-960f-4910-b4b9-25d5b68db3ca admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.783025 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1385/2776] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:43.787148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.797047 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d02725-5259-43fc-b322-4c4b69e119e9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.802546 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.802719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.802966 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.803248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] PATH_INFO: `/v3/projects/8962de0869254d17aa4253e5a5c6a059/users/c9dda10d7d294d1abb60e0d08d9c7344/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.818690 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8962de0869254d17aa4253e5a5c6a059, user_id=c9dda10d7d294d1abb60e0d08d9c7344, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.821398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.890599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1392/2777] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:43 2022] PUT /identity/v3/projects/8962de0869254d17aa4253e5a5c6a059/users/c9dda10d7d294d1abb60e0d08d9c7344/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:43.902035 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.902316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.902584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.934877 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] MFA Rules not processed for user `c9dda10d7d294d1abb60e0d08d9c7344`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:43.936172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.971075 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.003259 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1386/2778] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:44.008314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.030592 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.072522 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85929c-a926-481c-9680-61c85f546c53 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.077281 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.077890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.078178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.078463 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.079176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.081083 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.184285 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1393/2779] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:44 2022] POST /identity/v3/projects => generated 357 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:44.188848 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.200853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bb95a-2f54-49a6-b1d9-08e356c45414 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.260062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.260686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.260975 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.261316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.261968 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.263927 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.330306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1387/2780] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:44 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:44.334739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.345684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1348f-1e42-4222-8cf8-88aa3c095262 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.351570 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.352191 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.352478 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.352757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.353417 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.355138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.368780 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1394/2781] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:07:44.373092 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.383642 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484e802-4998-4947-a91b-09e3d870b055 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.388675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.389350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.389654 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.389919 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] PATH_INFO: `/v3/projects/0c20d54bbef84481baec311d7ebfdb94/users/67e8ec5b6ae5410eb038a37c0785d5e8/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.420776 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c20d54bbef84481baec311d7ebfdb94, user_id=67e8ec5b6ae5410eb038a37c0785d5e8, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.423504 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.484782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1388/2782] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:44 2022] PUT /identity/v3/projects/0c20d54bbef84481baec311d7ebfdb94/users/67e8ec5b6ae5410eb038a37c0785d5e8/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:44.489480 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.500099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07e7db-e48b-4362-982d-b1c6039887af admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.562034 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.562560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.562882 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.563153 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.563821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.565431 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d0b8-3daf-4e94-9bd1-cd4642869a6a admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.575733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1395/2783] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:44.580600 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.591203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3099e-6969-4622-860d-dd1747fd9a9e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.596108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.596801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.597088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.597362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] PATH_INFO: `/v3/projects/0c20d54bbef84481baec311d7ebfdb94/users/67e8ec5b6ae5410eb038a37c0785d5e8/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.599956 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c20d54bbef84481baec311d7ebfdb94, user_id=67e8ec5b6ae5410eb038a37c0785d5e8, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.602717 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.613507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1389/2784] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:44 2022] PUT /identity/v3/projects/0c20d54bbef84481baec311d7ebfdb94/users/67e8ec5b6ae5410eb038a37c0785d5e8/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:44.624732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.625042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.625322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.656642 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] MFA Rules not processed for user `67e8ec5b6ae5410eb038a37c0785d5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:44.658115 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.733398 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1396/2785] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:07:44.745499 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.747271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.819282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f26a9-d271-41a9-9547-9094c9845e64 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.825493 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.826550 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.827074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.827557 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.845109 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.848288 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.864311 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1390/2786] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:45.105741 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.116667 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5437-35ce-4304-a94e-93762d6b4e16 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.120419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.121055 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.121333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.121605 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.134204 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.179531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.183517 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260768a-4b3b-4e5d-98fb-6d72d0db4504 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.190168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1397/2787] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:54.856736 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:55.220858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=21 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.323612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.388624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7c5b-0dfe-41bc-ad50-55c85f12b23c service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.393668 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.394305 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.394586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.394891 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] PATH_INFO: `/v3/users/c9dda10d7d294d1abb60e0d08d9c7344` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.396031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9dda10d7d294d1abb60e0d08d9c7344)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.398114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.591273 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9dda10d7d294d1abb60e0d08d9c7344'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:04.684472 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9dda10d7d294d1abb60e0d08d9c7344'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:04.692167 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9dda10d7d294d1abb60e0d08d9c7344'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:04.697551 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9dda10d7d294d1abb60e0d08d9c7344'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:04.706007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9dda10d7d294d1abb60e0d08d9c7344'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:04.711869 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1391/2788] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:04 2022] DELETE /identity/v3/users/c9dda10d7d294d1abb60e0d08d9c7344 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:04.717299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.717806 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.717924 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.737148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:04.738372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.799250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1398/2789] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:04.810094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.811587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.870553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339b90-cc51-4d74-b294-40e94303e897 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.874322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.875085 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.875351 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.875625 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.885935 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.887422 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.897365 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1392/2790] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:05.361490 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.365484 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:05.377720 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77093c24-d381-4dac-b5be-d5eef58a2604 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.383372 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.384277 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.384439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.384669 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] PATH_INFO: `/v3/projects/8962de0869254d17aa4253e5a5c6a059` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.385956 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8962de0869254d17aa4253e5a5c6a059)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.388048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.514479 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8962de0869254d17aa4253e5a5c6a059', 'request_id': 'req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:05.519919 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8962de0869254d17aa4253e5a5c6a059', 'request_id': 'req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:05.650910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8962de0869254d17aa4253e5a5c6a059', 'request_id': 'req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:05.666453 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] The token cache is being invalidate because project 8962de0869254d17aa4253e5a5c6a059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:05.666835 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:05.667509 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:05.669009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1399/2791] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:05 2022] DELETE /identity/v3/projects/8962de0869254d17aa4253e5a5c6a059 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:05.673198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.683635 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07d7d8-dd79-4a6b-9c24-d463edc4ea21 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.707440 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.749603 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.750427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.750748 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.751027 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] PATH_INFO: `/v3/users/67e8ec5b6ae5410eb038a37c0785d5e8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.752171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e8ec5b6ae5410eb038a37c0785d5e8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.754104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.018282 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e8ec5b6ae5410eb038a37c0785d5e8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.136681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e8ec5b6ae5410eb038a37c0785d5e8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.144738 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e8ec5b6ae5410eb038a37c0785d5e8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.148000 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e8ec5b6ae5410eb038a37c0785d5e8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.159038 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ab77ab7b-d0db-434c-b5fb-b9227f5c428a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e8ec5b6ae5410eb038a37c0785d5e8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.165158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1393/2792] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:05 2022] DELETE /identity/v3/users/67e8ec5b6ae5410eb038a37c0785d5e8 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:06.641381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.642991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:06.656356 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6dae-7503-4fd9-9c66-aa51d9b55c39 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.723114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.723748 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.724025 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.724292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] PATH_INFO: `/v3/projects/0c20d54bbef84481baec311d7ebfdb94` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.725419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c20d54bbef84481baec311d7ebfdb94)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.727370 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.811733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:06.828462 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c20d54bbef84481baec311d7ebfdb94', 'request_id': 'req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.833779 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c20d54bbef84481baec311d7ebfdb94', 'request_id': 'req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.964817 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c20d54bbef84481baec311d7ebfdb94', 'request_id': 'req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.977105 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] The token cache is being invalidate because project 0c20d54bbef84481baec311d7ebfdb94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:06.977388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.978029 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-fb50c3e3-0cf8-47c8-8ac5-e6ad52a1c369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.980213 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1400/2793] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:06 2022] DELETE /identity/v3/projects/0c20d54bbef84481baec311d7ebfdb94 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:06.985138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.985656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.985656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.005834 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:07.007065 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.065851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1394/2794] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:07.074251 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.074542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.074854 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.095822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:07.097052 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.105760 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1401/2795] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:07.110327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.120585 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ffd27-6393-4125-9920-6a3f7bfef7d3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.125901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.126546 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.126846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.127146 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.127899 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:07.128361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.130063 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.142540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1395/2796] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:07.148099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.158048 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac7f4e-6d73-4292-964d-7e974c674c1e None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.162706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.163360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.163647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.163910 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.164679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.167236 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.268987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1402/2797] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:07.273443 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.284056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23140-5abe-4f49-96ec-cafa53fccaf7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.311880 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.344532 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.345062 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.345342 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.345707 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.346364 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.348361 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.456732 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1396/2798] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:07.461199 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.472324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcae4d-6577-4ef6-a7ad-eb3f3a291a79 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.476962 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.477628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.477921 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.478195 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.478884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.481602 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.490628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1403/2799] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:07.494965 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.505280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8da3-3ea2-4e30-8569-3210da607025 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.510527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.511157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.511480 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.511753 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.512426 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.514034 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbb23f-7ec2-49e3-a379-2eb0cf7a46cc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.524540 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1397/2800] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:07.529397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.539408 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114add9-b046-4469-a57e-8475cdf217b4 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.544830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.545074 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.545287 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.545559 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] PATH_INFO: `/v3/projects/582633efc03b43ddb9d1bcbb52ad0388/users/0f911ecae96b40848c1aa5f8646c7c22/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.561542 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582633efc03b43ddb9d1bcbb52ad0388, user_id=0f911ecae96b40848c1aa5f8646c7c22, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.564250 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.662122 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1404/2801] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:07 2022] PUT /identity/v3/projects/582633efc03b43ddb9d1bcbb52ad0388/users/0f911ecae96b40848c1aa5f8646c7c22/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:07.672553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.672833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.673104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.702247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] MFA Rules not processed for user `0f911ecae96b40848c1aa5f8646c7c22`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:07.703538 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.769121 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1398/2802] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:07.773730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.837263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79420f7a-8e19-4b60-bc22-0829769a4dc3 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.845070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.845676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.845953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.846219 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.846880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.848831 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.943512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1405/2803] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/projects => generated 363 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:07.947764 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.959773 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3c1f0-4828-49f2-965c-477dfc7373c8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.021431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.022178 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.022326 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.022602 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.023295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.025258 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.091846 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1399/2804] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:08.096051 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.107585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6a1bf-941d-401d-a30f-0f2e61c37e89 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.112813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.113439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.113723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.113999 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.114668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.116291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.128248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1406/2805] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:08.132624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.143528 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21a72-fb7b-4230-bdb1-c98788439c4a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.148348 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.149019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.149293 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.149567 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] PATH_INFO: `/v3/projects/6ce9a0bd03874f078e068729577cea8e/users/0eb9aaa16b1042d2a11927442ffd1a4d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.166387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce9a0bd03874f078e068729577cea8e, user_id=0eb9aaa16b1042d2a11927442ffd1a4d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.169599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.251895 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1400/2806] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:08 2022] PUT /identity/v3/projects/6ce9a0bd03874f078e068729577cea8e/users/0eb9aaa16b1042d2a11927442ffd1a4d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:08.256113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.266343 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817ae24-6f57-4a5e-8e45-cd76c076dcef admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.286151 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.325299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.325907 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.326185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.326452 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.327150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.328744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f1fa-7233-44a4-9f6b-8ef6eadf83f9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.338397 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1407/2807] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:08.342903 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.348453 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.354506 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b791c-1d66-43f4-8c85-bab863ab72f6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.359599 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.360491 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.360675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.360992 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] PATH_INFO: `/v3/projects/6ce9a0bd03874f078e068729577cea8e/users/0eb9aaa16b1042d2a11927442ffd1a4d/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.362983 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce9a0bd03874f078e068729577cea8e, user_id=0eb9aaa16b1042d2a11927442ffd1a4d, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.365630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.376863 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1401/2808] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:08 2022] PUT /identity/v3/projects/6ce9a0bd03874f078e068729577cea8e/users/0eb9aaa16b1042d2a11927442ffd1a4d/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:08.446056 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.446330 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.446614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.481324 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] MFA Rules not processed for user `0eb9aaa16b1042d2a11927442ffd1a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:08.482599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.559833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1408/2809] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:08.571300 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.572974 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.634396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25088c2-3b96-4efe-959c-4b7ead8176a5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.638166 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.638827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.639108 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.639379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.649002 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.650433 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.660837 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1402/2810] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:08.894069 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.904586 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1066e-b9dd-4ba6-8c1c-5160843ad784 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.908206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.908822 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.909154 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.909403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.968233 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.969679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df30b0-1df9-4ed5-bcc1-c448a3c6fcc7 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.978496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1409/2811] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:18.656393 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.974416 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=80 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.134698 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.204523 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f35c-cae2-465c-abd8-ac4b109e2b14 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.209022 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.209645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.209923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.210194 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] PATH_INFO: `/v3/users/0f911ecae96b40848c1aa5f8646c7c22` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.211322 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f911ecae96b40848c1aa5f8646c7c22)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.221332 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.407280 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f911ecae96b40848c1aa5f8646c7c22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.496204 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f911ecae96b40848c1aa5f8646c7c22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.504355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f911ecae96b40848c1aa5f8646c7c22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.505032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f911ecae96b40848c1aa5f8646c7c22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.517505 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f911ecae96b40848c1aa5f8646c7c22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.523854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1403/2812] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/users/0f911ecae96b40848c1aa5f8646c7c22 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:21.528374 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.528788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.528985 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.549935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:21.551286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.612634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1410/2813] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:21.623185 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.624833 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.683716 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fa0c5-750a-4160-90ed-4a0bb1166d75 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.687403 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.688074 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.688355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.688627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.697827 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.699292 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.708628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1404/2814] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:22.120686 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.122248 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.135148 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206e3f3-09a9-4ede-84ae-fd1b2bb91cff None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.140662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.141293 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.141577 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.148880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] PATH_INFO: `/v3/projects/582633efc03b43ddb9d1bcbb52ad0388` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.148880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] RBAC: Authorizing `identity:delete_project(project_id=582633efc03b43ddb9d1bcbb52ad0388)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.148880 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.274239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582633efc03b43ddb9d1bcbb52ad0388', 'request_id': 'req-87e86320-01f3-40b9-99a6-eefae68eccff'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.280555 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582633efc03b43ddb9d1bcbb52ad0388', 'request_id': 'req-87e86320-01f3-40b9-99a6-eefae68eccff'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.347812 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582633efc03b43ddb9d1bcbb52ad0388', 'request_id': 'req-87e86320-01f3-40b9-99a6-eefae68eccff'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.360811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] The token cache is being invalidate because project 582633efc03b43ddb9d1bcbb52ad0388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:22.361128 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.361775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.363407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1411/2815] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/projects/582633efc03b43ddb9d1bcbb52ad0388 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.367629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.378434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c578-3d84-44a0-983b-f4157e032bbe service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.397708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.437134 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.437853 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.438135 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.438397 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] PATH_INFO: `/v3/users/0eb9aaa16b1042d2a11927442ffd1a4d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.439594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb9aaa16b1042d2a11927442ffd1a4d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.441513 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.700379 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb9aaa16b1042d2a11927442ffd1a4d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.775969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb9aaa16b1042d2a11927442ffd1a4d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.783207 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb9aaa16b1042d2a11927442ffd1a4d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.783849 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb9aaa16b1042d2a11927442ffd1a4d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.796331 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5e8f1e67-2ac3-4e15-9201-e4822cdc0ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb9aaa16b1042d2a11927442ffd1a4d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.802138 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1405/2816] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/users/0eb9aaa16b1042d2a11927442ffd1a4d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.189283 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.190884 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.205023 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e86320-01f3-40b9-99a6-eefae68eccff admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.261961 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.262600 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.262908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.263183 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] PATH_INFO: `/v3/projects/6ce9a0bd03874f078e068729577cea8e` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.264286 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce9a0bd03874f078e068729577cea8e)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.266187 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.337647 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.354200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce9a0bd03874f078e068729577cea8e', 'request_id': 'req-ed946a82-88d4-4593-a531-6434457b3842'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.360268 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce9a0bd03874f078e068729577cea8e', 'request_id': 'req-ed946a82-88d4-4593-a531-6434457b3842'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.453087 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce9a0bd03874f078e068729577cea8e', 'request_id': 'req-ed946a82-88d4-4593-a531-6434457b3842'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.468513 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] The token cache is being invalidate because project 6ce9a0bd03874f078e068729577cea8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:23.468754 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.469567 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-ed946a82-88d4-4593-a531-6434457b3842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.471838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1412/2817] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:23 2022] DELETE /identity/v3/projects/6ce9a0bd03874f078e068729577cea8e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.477498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.478060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.478060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.499339 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.500572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.561228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1406/2818] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.576021 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.576333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.576604 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.597424 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.598805 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.606572 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1413/2819] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.611004 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.621040 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6311a-03e9-4ae0-8af2-1e35fb34c55b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.626180 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.628798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.628798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.628798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.628798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:23.628798 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.630199 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.642762 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1407/2820] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:23.647170 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.657662 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e0fb-283c-4e84-b333-d7cb32fd68e9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.662279 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.662935 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.663214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.663483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.664111 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.666075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.777618 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1414/2821] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/projects => generated 401 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:23.782413 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.793218 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1e14-2d2d-4285-83cf-8aff500ae7e4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.818803 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.854080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.854719 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.855005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.855279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.855923 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.857864 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.033645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1408/2822] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/users => generated 301 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.037643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.047866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc0cc-3eda-4ef6-a92c-4973825f7d88 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.053019 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.053631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.053908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.054178 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.055240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.057077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.067813 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1415/2823] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.073455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.084233 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3968-2122-405c-804c-969c7916b1c1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.088930 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.089533 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.089813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.090084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.090761 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.092388 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2fc0b-4015-4603-b8c8-281116509fe4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.102153 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1409/2824] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.106121 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.115904 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24f2d1-9c5c-43a8-b08d-7b07aec90a21 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.120367 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.121001 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.121278 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.122037 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] PATH_INFO: `/v3/projects/c9698145742b451798a99eb8793e2ed6/users/e4832d706fde443890cd095d35e3975c/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.136364 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9698145742b451798a99eb8793e2ed6, user_id=e4832d706fde443890cd095d35e3975c, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.139609 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.205790 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1416/2825] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/c9698145742b451798a99eb8793e2ed6/users/e4832d706fde443890cd095d35e3975c/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:24.216512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.216801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.217080 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.245117 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] MFA Rules not processed for user `e4832d706fde443890cd095d35e3975c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.247005 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.310782 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1410/2826] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:24.315262 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.375613 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449edbfd-2862-475e-adea-3fc8a30586b9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.380322 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.380928 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.381212 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.381488 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.382119 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.384067 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.480446 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1417/2827] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/projects => generated 399 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.485238 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.497781 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c930a-508d-4f24-ae6e-12f6795cb4df None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.562901 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.563431 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.563708 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.564058 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.564612 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.566553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.633378 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1411/2828] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/users => generated 306 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.637723 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.648202 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb071872-4f8b-4487-ac6a-a8b4b8cf9841 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.653840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.657650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.657650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.657650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.657650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.657650 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.668008 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1418/2829] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.672739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.682892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894e55-5f3e-4387-a08b-ef9cb63ad19f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.687503 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.688145 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.688427 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.688700 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] PATH_INFO: `/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a/users/4e923d5563de4fd79f40bce4eb1394b1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.703096 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f34d80704444fe9de16a71dfaf5b7a, user_id=4e923d5563de4fd79f40bce4eb1394b1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.705750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.764709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1412/2830] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a/users/4e923d5563de4fd79f40bce4eb1394b1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:24.768783 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.779933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e457-7321-4242-a4ab-c104cb9863d2 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.800309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.835787 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.836310 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.836589 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.836858 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.837543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.839160 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90b53c-c60c-4d21-ab66-8b9e4ccf8467 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.849511 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1419/2831] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:24.853516 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.859791 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.865191 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb12bc5-3777-4711-9076-5f4b7122c8c9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.869768 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.870423 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.870740 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.871019 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] PATH_INFO: `/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a/users/4e923d5563de4fd79f40bce4eb1394b1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.872977 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f34d80704444fe9de16a71dfaf5b7a, user_id=4e923d5563de4fd79f40bce4eb1394b1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.875709 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.885752 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1413/2832] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a/users/4e923d5563de4fd79f40bce4eb1394b1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:24.897145 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.902112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.902112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.926214 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] MFA Rules not processed for user `4e923d5563de4fd79f40bce4eb1394b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.927450 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.996229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1420/2833] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:25.007918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.009656 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:25.068043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f18043-0e13-4e15-b373-891f2dd463eb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.071809 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.072411 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.072692 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.072961 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.082619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.084114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.094366 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1414/2834] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:25.289333 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.299560 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed0660-5c12-4832-8649-5b76347db6c0 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.303138 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.303749 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.304032 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.304306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.361730 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.363143 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac58eb-41a8-4b73-bb0e-a27ed6c2e0ef service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.372460 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1421/2835] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:30.535733 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.603436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848f9bc-73e3-43de-9880-00b0b40c364b service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.607958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.608613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.608896 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.609192 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] PATH_INFO: `/v3/users/e4832d706fde443890cd095d35e3975c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.610327 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4832d706fde443890cd095d35e3975c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.612310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.830686 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4832d706fde443890cd095d35e3975c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.901876 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4832d706fde443890cd095d35e3975c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.909328 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4832d706fde443890cd095d35e3975c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.910079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4832d706fde443890cd095d35e3975c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.922464 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4832d706fde443890cd095d35e3975c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.928428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1415/2836] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:30 2022] DELETE /identity/v3/users/e4832d706fde443890cd095d35e3975c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:30.933482 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.934493 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.934622 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.953651 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:30.954893 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.993846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=100 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.012439 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1422/2837] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:31.022189 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.023628 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.037512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.082018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916c06d-42a1-4dab-a1a6-0a67d5de6dfa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.085702 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.086355 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.086633 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.086924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.096794 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.098245 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.107998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1416/2838] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:31.601684 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.603171 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.616762 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81790642-80e0-440b-9b47-f21f797e9bd9 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.621788 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.622461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.622866 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.623041 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] PATH_INFO: `/v3/projects/c9698145742b451798a99eb8793e2ed6` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.624746 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9698145742b451798a99eb8793e2ed6)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.626739 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.749156 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9698145742b451798a99eb8793e2ed6', 'request_id': 'req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.754531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9698145742b451798a99eb8793e2ed6', 'request_id': 'req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.839260 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9698145742b451798a99eb8793e2ed6', 'request_id': 'req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.852077 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] The token cache is being invalidate because project c9698145742b451798a99eb8793e2ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:31.852352 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.852969 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.854492 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1423/2839] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:31 2022] DELETE /identity/v3/projects/c9698145742b451798a99eb8793e2ed6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:31.858712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.868524 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9708b-32f7-4b8d-a35f-db07b3e941f6 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.927617 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.928251 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.928535 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.928801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] PATH_INFO: `/v3/users/4e923d5563de4fd79f40bce4eb1394b1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.929969 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e923d5563de4fd79f40bce4eb1394b1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.932236 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.133922 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e923d5563de4fd79f40bce4eb1394b1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.200350 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e923d5563de4fd79f40bce4eb1394b1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.208094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e923d5563de4fd79f40bce4eb1394b1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.210933 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e923d5563de4fd79f40bce4eb1394b1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.225271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4c64cfa6-1783-49c7-8b31-892a8fa15a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e923d5563de4fd79f40bce4eb1394b1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.231618 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1417/2840] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:31 2022] DELETE /identity/v3/users/4e923d5563de4fd79f40bce4eb1394b1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:32.664011 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.667301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:32.679053 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d1ec6-39b7-4ea0-9a2e-21b4f9fed6fd admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.738676 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.739305 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.739585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.739853 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] PATH_INFO: `/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.740963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f34d80704444fe9de16a71dfaf5b7a)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.742900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.875744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f34d80704444fe9de16a71dfaf5b7a', 'request_id': 'req-c849955e-1caa-4208-b68b-57fb3cbfbc58'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.881483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f34d80704444fe9de16a71dfaf5b7a', 'request_id': 'req-c849955e-1caa-4208-b68b-57fb3cbfbc58'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.959070 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f34d80704444fe9de16a71dfaf5b7a', 'request_id': 'req-c849955e-1caa-4208-b68b-57fb3cbfbc58'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.967235 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.971789 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] The token cache is being invalidate because project e9f34d80704444fe9de16a71dfaf5b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:32.972086 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.972732 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c849955e-1caa-4208-b68b-57fb3cbfbc58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.974222 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1424/2841] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:32 2022] DELETE /identity/v3/projects/e9f34d80704444fe9de16a71dfaf5b7a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:32.979088 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.979584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.979706 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.999424 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:33.000629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.030619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.065018 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1418/2842] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:33.074085 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.074389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.074643 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.096229 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:33.097457 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.105634 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1425/2843] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:33.110630 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.120539 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162c8df-700f-44e4-8628-8bd9a876a03c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.125416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.126032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.126316 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.126592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.127284 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.127588 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.129247 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.143937 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1419/2844] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.148974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.159112 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284b894-cc57-4e6d-881e-dce156b1de66 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.163843 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.164441 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.164716 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.164991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.165623 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.167578 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.270314 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1426/2845] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:33.274808 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.285820 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf445a6f-75d2-4572-ac76-6857233f601b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.344249 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.344854 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.345129 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.345396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.346010 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.347958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.416759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1420/2846] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:33.421046 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.433082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee30bb-6f31-4b6f-a19b-bdbc2f29bfd9 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.437757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.438361 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.438644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.438946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.439591 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.441182 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.451512 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1427/2847] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:33.455581 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.466295 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a555d2-2f9f-4eaa-b82f-ae879a2c94aa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.471023 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.471620 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.471938 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.472212 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.472855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.474473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca83e1b-eb51-42b6-a38f-2b3ae717b3d8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.485690 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1421/2848] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:33.490124 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.500188 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f6ade-dbab-46bb-a1eb-aecd5b433610 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.505256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.505932 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.506300 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.506554 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] PATH_INFO: `/v3/projects/fe1399fcfdae4e6c9175a7d08f36a11f/users/cd77c9569da8423faa4f58e62eec9a64/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.522291 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1399fcfdae4e6c9175a7d08f36a11f, user_id=cd77c9569da8423faa4f58e62eec9a64, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.524963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.570914 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1428/2849] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:33 2022] PUT /identity/v3/projects/fe1399fcfdae4e6c9175a7d08f36a11f/users/cd77c9569da8423faa4f58e62eec9a64/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:33.583234 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.583525 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.583805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.611428 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] MFA Rules not processed for user `cd77c9569da8423faa4f58e62eec9a64`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:33.612645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.677960 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1422/2850] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:33.682818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.744733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e2de2-0999-4667-ac37-4772572ad857 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.750185 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.750840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.751127 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.751403 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.752042 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.754240 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.846807 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1429/2851] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:33.851239 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.862276 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e873-ed7f-49d9-9cdb-443a720fb003 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.890898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.921999 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.925855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.925855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.925855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.925855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.925855 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.011006 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1423/2852] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:34.014809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.024995 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5911-1133-45a4-ba3d-aac69ad13d3c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.029838 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.030474 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.030799 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.031076 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.031758 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.033318 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.044476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1430/2853] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:34.048792 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.059498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d30057-f723-496d-ac97-81f7c06b5128 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.064064 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.064710 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.064990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.065262 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] PATH_INFO: `/v3/projects/eaf48faa4da347878872955eb3f0534a/users/02d2d37592364ca3965d100f6ef125b6/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.080211 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf48faa4da347878872955eb3f0534a, user_id=02d2d37592364ca3965d100f6ef125b6, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.082886 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.159097 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1424/2854] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:34 2022] PUT /identity/v3/projects/eaf48faa4da347878872955eb3f0534a/users/02d2d37592364ca3965d100f6ef125b6/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:34.163132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.173913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56457c7d-48eb-423b-9891-bcca1b619422 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.233054 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.233677 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.233963 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.234241 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.234920 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.236533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ee533-e192-426c-8f47-621e05aefb40 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.247590 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1431/2855] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:34.251892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.262998 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f619-558d-4131-9705-d07fcadb2a29 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.267735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.268429 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.268730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.269003 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] PATH_INFO: `/v3/projects/eaf48faa4da347878872955eb3f0534a/users/02d2d37592364ca3965d100f6ef125b6/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.270946 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf48faa4da347878872955eb3f0534a, user_id=02d2d37592364ca3965d100f6ef125b6, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.273572 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.284224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1425/2856] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:34 2022] PUT /identity/v3/projects/eaf48faa4da347878872955eb3f0534a/users/02d2d37592364ca3965d100f6ef125b6/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:34.295303 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.295599 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.295881 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.304059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.325331 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] MFA Rules not processed for user `02d2d37592364ca3965d100f6ef125b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:34.326552 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:34.399582 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1432/2857] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:34.411084 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.412730 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:34.469984 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac152e-ed13-4b64-a9df-cf228d0d89cf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.473682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.474306 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.474586 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.474887 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.485024 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.486507 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50502033-f003-4aad-8019-bff7d6386ef8 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.496125 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1426/2858] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:34.991436 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.054543 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3e730-49cc-4ee2-b14b-939bae4143f7 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.059627 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.060263 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.060547 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.060824 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] PATH_INFO: `/v3/users/cd77c9569da8423faa4f58e62eec9a64` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.062215 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd77c9569da8423faa4f58e62eec9a64)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.064166 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.258377 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd77c9569da8423faa4f58e62eec9a64'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:35.258982 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd77c9569da8423faa4f58e62eec9a64'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:35.353733 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd77c9569da8423faa4f58e62eec9a64'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:35.361135 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd77c9569da8423faa4f58e62eec9a64'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:35.373385 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd77c9569da8423faa4f58e62eec9a64'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:35.379234 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1433/2859] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:34 2022] DELETE /identity/v3/users/cd77c9569da8423faa4f58e62eec9a64 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:35.384739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.384739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.384739 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.404067 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:35.405255 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:35.466934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1427/2860] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:35.476846 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.478309 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:35.490335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.542123 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55f95-cfc2-43f0-9fcb-7b88dec70b5c admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.546890 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.547510 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.547792 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.548064 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.559395 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.562335 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.575018 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1434/2861] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:35.994857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.999990 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.008060 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d940f-b888-458f-be2b-54e0adfdf770 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.013278 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.013909 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.014183 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.014455 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] PATH_INFO: `/v3/projects/fe1399fcfdae4e6c9175a7d08f36a11f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.015805 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1399fcfdae4e6c9175a7d08f36a11f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.017671 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.040501 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.114046 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1399fcfdae4e6c9175a7d08f36a11f', 'request_id': 'req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.119670 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1399fcfdae4e6c9175a7d08f36a11f', 'request_id': 'req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.187870 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1399fcfdae4e6c9175a7d08f36a11f', 'request_id': 'req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.199386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] The token cache is being invalidate because project fe1399fcfdae4e6c9175a7d08f36a11f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:36.200434 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.201553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.203093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1428/2862] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:35 2022] DELETE /identity/v3/projects/fe1399fcfdae4e6c9175a7d08f36a11f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:36.207137 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.217389 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91a1eb-2ca6-48e0-a332-fcb23858f352 service neutron] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.274830 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.275466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.275753 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.276030 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] PATH_INFO: `/v3/users/02d2d37592364ca3965d100f6ef125b6` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.277150 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d2d37592364ca3965d100f6ef125b6)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.279097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.459282 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d2d37592364ca3965d100f6ef125b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.459946 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d2d37592364ca3965d100f6ef125b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.525585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d2d37592364ca3965d100f6ef125b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.533036 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d2d37592364ca3965d100f6ef125b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.545301 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-d9e96c3a-ad3d-4b9e-959b-8a7dd6518be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d2d37592364ca3965d100f6ef125b6'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.551503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1435/2863] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:36 2022] DELETE /identity/v3/users/02d2d37592364ca3965d100f6ef125b6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:36.960857 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.962387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.978555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4355c-f7f3-4abc-b849-cd9631d0b6e0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.037875 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.038508 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.038819 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.039093 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] PATH_INFO: `/v3/projects/eaf48faa4da347878872955eb3f0534a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.040256 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf48faa4da347878872955eb3f0534a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.042162 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.194451 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf48faa4da347878872955eb3f0534a', 'request_id': 'req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:37.200102 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf48faa4da347878872955eb3f0534a', 'request_id': 'req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:37.290874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf48faa4da347878872955eb3f0534a', 'request_id': 'req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:37.303324 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] The token cache is being invalidate because project eaf48faa4da347878872955eb3f0534a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:37.303611 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:37.306386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d61fa7fb-bc6f-43a8-8b32-3cad529167c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:37.306386 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1429/2864] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:36 2022] DELETE /identity/v3/projects/eaf48faa4da347878872955eb3f0534a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:37.311079 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.311585 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.311697 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.331075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.332272 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.358987 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.392856 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1436/2865] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:37.400851 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.401171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.401448 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.421720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.423037 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.430918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1430/2866] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:37.435545 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.445239 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb3f2-3e4a-42ad-b208-45a6a59258ff None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.450837 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.451438 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.451719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.451992 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.452631 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:37.452933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.454531 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.467302 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1437/2867] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:37.471759 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.481081 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b21667-348b-4097-9924-6dfd792756ee None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.485829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.486425 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.486735 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.487009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.487627 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.489578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.596265 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1431/2868] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:37.600483 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.610409 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1bd7f-825c-44e6-a6ed-f8cfe3a42868 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.667014 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.667628 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.667908 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.668207 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.669003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.670940 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.742378 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1438/2869] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:37.747801 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.758696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d540532-0db6-4d51-b32c-cfc82a81725c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.764050 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.764675 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.764948 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.765224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.765874 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.767522 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.769958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.778824 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1432/2870] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:37.785487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.796297 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef607c-47d8-4227-b916-36aa8a315a47 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.801157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.801773 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.802059 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.805668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.805668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.805668 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d9382-2a1c-40c8-ac07-d904de5c6e51 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.817141 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1439/2871] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:37.821296 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.832264 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a865-2a4f-4f75-a866-1f355794da2d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.837190 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.837980 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.838203 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.838479 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] PATH_INFO: `/v3/projects/ca8eac2ff5ca40f3ba722a8bebc2c52f/users/f608c264d962436fb8c50110e27119d0/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.854750 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8eac2ff5ca40f3ba722a8bebc2c52f, user_id=f608c264d962436fb8c50110e27119d0, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.858176 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.932319 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1433/2872] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:37 2022] PUT /identity/v3/projects/ca8eac2ff5ca40f3ba722a8bebc2c52f/users/f608c264d962436fb8c50110e27119d0/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:37.944801 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.945095 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.945376 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.973414 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] MFA Rules not processed for user `f608c264d962436fb8c50110e27119d0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.974638 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:38.038544 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1440/2873] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:38.050056 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.051793 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:38.108932 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343f300-fae1-4084-a312-7de23073f7fc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.112498 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd94c7f51f5d5478f858a8eb47cd89257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.113094 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.113367 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.113636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.122353 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.123958 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc958-e368-41f6-94eb-d274c0b6103b service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.132541 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1434/2874] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:48.035745 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.128645 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.476526 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.543548 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30a72-b276-44c8-8031-89c4f507b3c6 None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.548430 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.549091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.549379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.549657 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] PATH_INFO: `/v3/users/f608c264d962436fb8c50110e27119d0` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.550834 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f608c264d962436fb8c50110e27119d0)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.552900 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.787033 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f608c264d962436fb8c50110e27119d0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:56.787644 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f608c264d962436fb8c50110e27119d0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:56.852114 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f608c264d962436fb8c50110e27119d0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:56.860757 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f608c264d962436fb8c50110e27119d0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:56.873455 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f608c264d962436fb8c50110e27119d0'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:56.882550 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1441/2875] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:56 2022] DELETE /identity/v3/users/f608c264d962436fb8c50110e27119d0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:56.887436 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.888313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.888313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.907681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:56.908924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:56.994345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1435/2876] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:57.005231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.007026 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.071487 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eed754-0074-4a98-b2fd-506e7d28d084 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.075168 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.075800 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.076082 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.076358 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.086132 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.087645 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c34a6-ed40-4f41-b1ba-3ac163fe1f30 service neutron] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.100459 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1442/2877] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:57.527412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.527412 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.537113 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216722-8384-4fe1-8899-278fcd5902a4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.542416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.543116 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.543387 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.543829 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] PATH_INFO: `/v3/projects/ca8eac2ff5ca40f3ba722a8bebc2c52f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.545157 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8eac2ff5ca40f3ba722a8bebc2c52f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.547171 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.646594 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8eac2ff5ca40f3ba722a8bebc2c52f', 'request_id': 'req-04968e61-eec3-4e9d-8745-aa4867dddcab'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:57.652279 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8eac2ff5ca40f3ba722a8bebc2c52f', 'request_id': 'req-04968e61-eec3-4e9d-8745-aa4867dddcab'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:57.731435 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8eac2ff5ca40f3ba722a8bebc2c52f', 'request_id': 'req-04968e61-eec3-4e9d-8745-aa4867dddcab'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:57.743917 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] The token cache is being invalidate because project ca8eac2ff5ca40f3ba722a8bebc2c52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:57.744228 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:57.744907 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-04968e61-eec3-4e9d-8745-aa4867dddcab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:57.746396 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1436/2878] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:57 2022] DELETE /identity/v3/projects/ca8eac2ff5ca40f3ba722a8bebc2c52f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:57.751329 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.751840 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.751974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.773000 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:57.774206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.793005 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.837679 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1443/2879] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:57.846246 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.846527 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.846920 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.867359 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:57.868592 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.876812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1437/2880] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:57.881306 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.891133 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfca34-72ec-4635-b24d-2c4f30a4328b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.896466 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.897383 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.898075 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.898209 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.898925 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:57.899227 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.900833 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.913336 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1444/2881] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:57.919898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.929563 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca632f-6933-4b0c-b7bd-205f70d7d3fd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.934720 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.935333 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.935653 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.935928 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.936587 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.938993 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.047291 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1438/2882] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:58.052091 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.063164 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4420a8-4f71-4dea-bd34-907310f88b19 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.120298 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.120810 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.121088 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.121362 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.121973 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.123898 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.234719 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1445/2883] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:58 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:58.239158 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.250043 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3b32-45ec-4c4c-9412-7fdfe385d7ef admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.255363 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.261924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.261924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.261924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.261924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.261924 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.269584 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1439/2884] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:58.273811 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.284419 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185386d5-9e73-4f9c-b258-551c4ee694d6 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.289176 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.289819 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.290100 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.290382 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.291099 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.292861 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7189ba2-764b-4fac-9391-8d975fa8f658 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.304503 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1446/2885] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:58.309468 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.320484 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc3971-5fff-4bc0-bea3-bfa85e79e767 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.325124 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.325790 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.326071 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.326345 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] PATH_INFO: `/v3/projects/eccdd3b0ae2c45108605cd3e8c391773/users/6ad9e2bf41c544b6a23bdb0b7058b161/roles/4eed1433d0c54215a270109801b14d73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.342313 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccdd3b0ae2c45108605cd3e8c391773, user_id=6ad9e2bf41c544b6a23bdb0b7058b161, role_id=4eed1433d0c54215a270109801b14d73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.345020 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.435209 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1440/2886] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:58 2022] PUT /identity/v3/projects/eccdd3b0ae2c45108605cd3e8c391773/users/6ad9e2bf41c544b6a23bdb0b7058b161/roles/4eed1433d0c54215a270109801b14d73 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:58.447683 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.454991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.454991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.475189 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] MFA Rules not processed for user `6ad9e2bf41c544b6a23bdb0b7058b161`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:58.476388 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:58.539476 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1447/2887] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:58.544133 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.566104 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:58.605289 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30cb42-c11a-4ee5-9f83-71ff8f1819c1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.609902 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.611031 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.611310 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.611577 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.612198 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.614110 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.709466 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1441/2888] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:58 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:58.713595 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.724744 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047ebc6-33af-4a8a-87d2-d82ed224cd4b None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.737615 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:58.780614 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.781312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.781495 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.781771 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] PATH_INFO: `/v3/users` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.782933 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.784845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.863256 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1448/2889] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:58 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:58.867416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.878032 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6905-a8d0-4c08-94a1-a93036625c91 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.882715 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.883315 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.883619 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.883962 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.884609 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.886208 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.897918 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1442/2890] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:58.902181 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.912798 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f593ff9-63d5-459d-9d49-88ea7d3b6af5 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.917360 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.918013 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.918345 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.918616 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] PATH_INFO: `/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20/users/16738593e6874da19235c1fb449b87f1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.934177 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f07a1dbd8be4085ba16b5faad5efc20, user_id=16738593e6874da19235c1fb449b87f1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.936953 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.005775 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1449/2891] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:58 2022] PUT /identity/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20/users/16738593e6874da19235c1fb449b87f1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:59.009831 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.020362 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df397f-6b29-4500-ae75-8326d2cc8cf1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.083307 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.083934 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.084220 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.084494 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.085139 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.086780 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dae670-61b9-4991-88b0-f63e364dc252 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.097931 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1443/2892] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:59.102381 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.112876 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bac04c-9f10-4ba3-8c03-73faf7daf7cc admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.117534 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.118206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.118489 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.118827 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] PATH_INFO: `/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20/users/16738593e6874da19235c1fb449b87f1/roles/3f587a5233884fd6861867344e2c084b` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.120706 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f07a1dbd8be4085ba16b5faad5efc20, user_id=16738593e6874da19235c1fb449b87f1, role_id=3f587a5233884fd6861867344e2c084b)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.123379 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.133930 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1450/2893] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:59 2022] PUT /identity/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20/users/16738593e6874da19235c1fb449b87f1/roles/3f587a5233884fd6861867344e2c084b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:59.144789 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.145087 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.145357 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.172797 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] MFA Rules not processed for user `16738593e6874da19235c1fb449b87f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.174042 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.242029 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1444/2894] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:59.253225 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.255007 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.311461 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740835-8927-4860-b1fd-e4ded3d5ba8e admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.315097 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a67c22370884cefb7795cc9eb8b2f8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.315698 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] REQUEST_METHOD: `GET` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.315979 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.316252 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.371231 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.372675 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e09c8-6a73-46e7-b5f7-2c4442668559 service cinder] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.381339 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1451/2895] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:02.370777 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.387783 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.434007 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770d36a-1c77-4e59-85e0-863cedb536a4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.439224 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.439884 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.440172 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.440444 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] PATH_INFO: `/v3/users/6ad9e2bf41c544b6a23bdb0b7058b161` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.441578 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad9e2bf41c544b6a23bdb0b7058b161)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.443553 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.648712 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad9e2bf41c544b6a23bdb0b7058b161'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:02.725549 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad9e2bf41c544b6a23bdb0b7058b161'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:02.733271 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad9e2bf41c544b6a23bdb0b7058b161'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:02.735555 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad9e2bf41c544b6a23bdb0b7058b161'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:02.747009 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad9e2bf41c544b6a23bdb0b7058b161'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:02.752971 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1445/2896] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:02 2022] DELETE /identity/v3/users/6ad9e2bf41c544b6a23bdb0b7058b161 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:02.757485 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] REQUEST_METHOD: `POST` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.758045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.758045 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.778149 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.auth.core [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] MFA Rules not processed for user `9e24fe7edd574919ac552aa8b7b7cbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:02.779407 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:02.798311 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.839292 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1452/2897] 10.222.0.64 () {58 vars in 958 bytes} [Wed Aug 24 09:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:02.850114 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.851593 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:02.911230 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b34a-86dd-41aa-afef-31e50c6b7dea admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.914744 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06a32a4ecd71495785ec867dafc1c8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.915349 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.915629 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.915898 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.925821 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.927297 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.937048 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1446/2898] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:03.397422 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:03.399113 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:03.412541 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5985a-76c9-469b-9396-c3bb9fde604c None None] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.417533 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.418249 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.418496 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.418809 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] PATH_INFO: `/v3/projects/eccdd3b0ae2c45108605cd3e8c391773` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.420003 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccdd3b0ae2c45108605cd3e8c391773)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.421974 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.595929 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccdd3b0ae2c45108605cd3e8c391773', 'request_id': 'req-7547dd25-26be-40ec-a66c-36930975ce86'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:03.601714 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccdd3b0ae2c45108605cd3e8c391773', 'request_id': 'req-7547dd25-26be-40ec-a66c-36930975ce86'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:03.680206 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccdd3b0ae2c45108605cd3e8c391773', 'request_id': 'req-7547dd25-26be-40ec-a66c-36930975ce86'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:03.692911 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] The token cache is being invalidate because project eccdd3b0ae2c45108605cd3e8c391773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:03.693200 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:03.693863 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:03.695426 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1453/2899] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:03 2022] DELETE /identity/v3/projects/eccdd3b0ae2c45108605cd3e8c391773 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:03.699636 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:03.709613 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea965fb-0f21-42d8-ad92-b44a30e02fc6 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.743797 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:03.767510 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.768143 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.768416 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.768681 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] PATH_INFO: `/v3/users/16738593e6874da19235c1fb449b87f1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.769963 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16738593e6874da19235c1fb449b87f1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.771892 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.974728 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16738593e6874da19235c1fb449b87f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.057252 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16738593e6874da19235c1fb449b87f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.065812 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16738593e6874da19235c1fb449b87f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.066482 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16738593e6874da19235c1fb449b87f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.078696 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16738593e6874da19235c1fb449b87f1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.085624 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1447/2900] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:03 2022] DELETE /identity/v3/users/16738593e6874da19235c1fb449b87f1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:04.756991 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Authenticating user token {{(pid=93428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.758821 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.fernet_utils [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:04.777913 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547dd25-26be-40ec-a66c-36930975ce86 admin admin] Validating token access rules against request {{(pid=93428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.847157 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e24fe7edd574919ac552aa8b7b7cbf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6994c6fdbe24b2e98be4db89dbc57c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.848312 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.848594 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.848870 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] PATH_INFO: `/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20` {{(pid=93428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.850044 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f07a1dbd8be4085ba16b5faad5efc20)` {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.852104 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] RBAC: Authorization granted {{(pid=93428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.961818 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f07a1dbd8be4085ba16b5faad5efc20', 'request_id': 'req-48071bcd-e3b6-4b35-ab9d-d258380b33f4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:04.963139 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.968491 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f07a1dbd8be4085ba16b5faad5efc20', 'request_id': 'req-48071bcd-e3b6-4b35-ab9d-d258380b33f4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:05.074845 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f07a1dbd8be4085ba16b5faad5efc20', 'request_id': 'req-48071bcd-e3b6-4b35-ab9d-d258380b33f4'} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:05.088009 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] The token cache is being invalidate because project 4f07a1dbd8be4085ba16b5faad5efc20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:05.088299 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:05.088965 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG keystone.notifications [None req-48071bcd-e3b6-4b35-ab9d-d258380b33f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:05.090530 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: [pid: 93428|app: 0|req: 1454/2901] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:04 2022] DELETE /identity/v3/projects/4f07a1dbd8be4085ba16b5faad5efc20 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:14.081152 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:15.154784 dv-cinder-852211-5-iscsi devstack@keystone.service[93428]: DEBUG dbcounter [-] [93428] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=93428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:15.651375 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.652817 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.731682 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3b8-64fe-454a-ba98-4d02702bad1c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.798574 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '440669b078bd474d955d2faea0dbe859', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8af04616519544be965cfabfe2f8dbbb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.799223 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.799512 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.799788 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.801569 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.814335 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.816079 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dec059-bb46-4df4-9dfc-e709ae901a47 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.827473 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1448/2902] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:25.822813 dv-cinder-852211-5-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=58 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}